A lightweight aerial drone dangerous action detection method and system

Abstract

The present invention relates to a lightweight method and system for detecting dangerous aerial drone movements. The method comprises: constructing a lightweight target detection network through a combination of depthwise separable convolution and dynamic resolution adjustment to locate drone targets in captured video frames; predicting and tracking the target's motion state through Kalman filtering in conjunction with drone target positioning; constructing a target memory library through a dynamic screening mechanism; and retrieving the most recent high-confidence feature from the target memory library to update the drone's target state when target association matching fails during tracking; and analyzing the drone's motion characteristics to detect dangerous drone movements. The present invention addresses the problems of high computing resource requirements, insufficient real-time performance, and poor generalization in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method and system for detecting dangerous actions of lightweight aerial drones. Background Art

[0002] With the rapid development of drone technology, drones have been widely used in fields such as express delivery and logistics, emergency rescue, environmental monitoring, and military reconnaissance. However, during flight, drones can experience dangerous maneuvers such as loss of control, violent oscillations, and abnormal acceleration due to complex and changing external environments, such as sudden strong winds, obstacles, and signal interference, as well as operational errors and system failures. These dangerous maneuvers not only affect the drone's operational stability but also pose serious threats to people, property, and the environment on the ground.

[0003] Existing methods for detecting dangerous drone movements typically rely on high-precision sensor fusion or complex deep learning models. While these methods offer high detection accuracy, they suffer from the following issues:

[0004] (1) High computing resource consumption: Most detection algorithms require high-performance computing devices or cloud support, which increases the complexity of detection hardware design and operating costs.

[0005] (2) Latency problem: Some methods require uploading data to the cloud for processing, resulting in the inability to return detection results in real time, and are not suitable for real-time detection needs.

[0006] (3) Insufficient model generalization: Many algorithms are highly dependent on specific environments or tasks and are difficult to adapt to different drone types or flight scenarios.

[0007] Therefore, it is of great research significance to design a lightweight, real-time, efficient and low-energy consumption method for detecting dangerous actions of drones. Summary of the Invention

[0008] The purpose of the present invention is to provide a lightweight method and system for detecting dangerous actions of aerial drones, so as to solve the problems of high computing resource requirements, insufficient real-time performance and poor generalization in the prior art.

[0009] The technical solutions for achieving the purpose of the present invention are:

[0010] A method for detecting dangerous actions of lightweight aerial drones, comprising:

[0011] Step 1: By combining depthwise separable convolution and dynamic resolution adjustment, a lightweight target detection network is constructed to locate the drone target in the collected video frames.

[0012] Step 2: Combined with the UAV target positioning, the target's motion state is predicted and tracked through Kalman filtering;

[0013] Step 3: Build a target memory library through a dynamic screening mechanism. When target association matching fails during tracking, retrieve the most recent high-confidence feature from the target memory library to restore the drone's target state.

[0014] Step 4: Analyze the motion characteristics of the drone target and detect dangerous drone actions.

[0015] Furthermore, the lightweight target detection network includes a feature extraction layer, a hierarchical attention layer, and an output layer. The drone target positioning on the collected video frames specifically includes:

[0016] Step 1.1: The feature extraction layer uses a lightweight backbone network to extract multi-scale features of the video frame and obtain a feature map.

[0017] Step 1.2: Hierarchical attention introduces channel attention and spatial attention into the feature map to enhance the feature map;

[0018] In step 1.3, based on the enhanced feature map, the output layer generates the target detection box, category, and confidence score through the lightweight target detection head. The target with a confidence score exceeding the confidence threshold is regarded as the target detection result.

[0019] Furthermore, the multi-scale features of the video frame extracted in step 1.1 are:

[0020] ;

[0021] in, is the feature of the i-th layer of the network, is a depth-wise separable convolution, is point-wise convolution, I t is the video frame;

[0022] The channel attention and spatial attention in step 1.2 are:

[0023] ;

[0024] ;

[0025] in, is the feature of the i-th layer of the network, AvgPool represents the average pooling operation, W c 、W s Represents weight, Conv represents convolution operation, Activated for Hard-Swish.

[0026] Furthermore, in step 2, the motion state of the target is predicted and tracked by using Kalman filtering, which specifically includes:

[0027] Step 2.1, initialize the state vector of each detected target;

[0028] Step 2.2, initialize the covariance matrix;

[0029] Step 2.3: Based on the initialized state vector, the Kalman filter is used to predict the next state of the target.

[0030] In step 2.4, the matching score is calculated based on the intersection over union (IoU) of the detected state and the predicted state. If the matching score is greater than the IoU threshold, step 2.5 is executed. Otherwise, step 3 is executed to retrieve the most recent high-confidence feature from the target memory and restore the drone's target state.

[0031] In step 2.5, based on the initialized state vector and covariance matrix, the target state is updated using the detection state through the Kalman filter.

[0032] Furthermore, the state at the next moment in step 2.3 is:

[0033] ;

[0034] Where F is the state transfer matrix;

[0035] Step 2.5 updates the target status to:

[0036] ;

[0037] ;

[0038] in, is the observation value of the detection box, is the observation matrix, which is used to map the state to the observation space, is the Kalman gain, is the predicted state covariance matrix, is the measurement noise covariance matrix, The status at the previous moment.

[0039] Furthermore, in step 3, a target memory library is constructed through a dynamic screening mechanism, which specifically includes:

[0040] Calculate the comprehensive score by matching the score and the target score;

[0041] Based on the comprehensive score, historical features with high confidence that meet the requirements are saved as the target memory library.

[0042] Furthermore, the comprehensive score is:

[0043] ;

[0044] ;

[0045] ;

[0046] in, is the matching score, The detected target detection box, IoU is the intersection over union ratio, is the predicted target state, is the target score, which indicates the probability of the target existing. is the confidence score of the kth frame, is the smoothing parameter, 、 is the weight coefficient;

[0047] High confidence historical characteristics satisfy:

[0048] ;

[0049] in, The features of the i-th target at time k, is the scoring threshold, is the maximum number of memory frames, and t is the frame number.

[0050] Furthermore, the step 4 specifically includes:

[0051] Step 4.1, calculate the three-dimensional space coordinates of the target in the camera coordinate system;

[0052] Step 4.2, calculate the velocity and acceleration based on the three-dimensional space coordinates;

[0053] Step 4.3, calculate the target flight path angle based on the velocity;

[0054] Step 4.4: Based on the velocity, acceleration, and flight path angle, a feature sequence within a time window is constructed, and the dynamic time warping algorithm is used to match the similarity between the current time feature sequence and the template feature sequence;

[0055] In step 4.5, based on the similarity, a comprehensive anomaly score is obtained. If the comprehensive anomaly score is greater than the threshold, it is considered a dangerous action.

[0056] Furthermore, the comprehensive anomaly score in step 4.5 is:

[0057] ;

[0058] ;

[0059] in, is the weight parameter, is the time interval, is the velocity increment, is the acceleration increment, DTW is the dynamic time warping function, is the time-aligned path, is the kth feature of the current sequence, is the alignment feature of the template sequence.

[0060] A lightweight aerial drone dangerous action detection system, comprising:

[0061] The target positioning module builds a lightweight target detection network through the combination of depth-separable convolution and dynamic resolution adjustment to perform drone target positioning on the collected video frames.

[0062] The motion modeling and tracking module, combined with UAV target positioning, predicts and tracks the target's motion state through Kalman filtering;

[0063] The motion perception memory optimization module builds a target memory library through a dynamic screening mechanism. During the tracking process, if the target association matching fails, the most recent high-confidence feature is retrieved from the target memory library to restore the drone's target status.

[0064] The motion characteristics analysis module analyzes the motion characteristics of drone targets and detects dangerous drone actions.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] 1. Overall optimization of lightweight design

[0067] Depthwise Separable Convolution: The use of a lightweight network structure in the target detection stage significantly reduces computational costs, making it suitable for resource-constrained devices. Traditional target detection and tracking algorithms often pursue accuracy while ignoring efficiency, making them difficult to run in real time in embedded or low-computing scenarios. The combination of depthwise separable convolution and dynamic resolution adjustment achieves lightweight and efficient detection capabilities.

[0068] 2. Motion perception memory optimization mechanism

[0069] The memory selection method introduces a hybrid scoring function to dynamically build a memory library. In the event of target occlusion or temporary loss, high-confidence historical features are selected for target recovery. Traditional tracking algorithms typically use fixed windows or time series to manage memory, which may store low-quality redundant information. This algorithm significantly reduces error propagation and improves long-term tracking stability through motion-aware optimization.

[0070] 3. The design of the present invention has obvious advantages in low-resource scenarios, achieving motion tracking and dynamic behavior analysis with higher robustness and response speed, and providing effective technical support for drone safety monitoring and urban airspace management. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 Flow chart of the method of the present invention.

[0072] Figure 2 This is the rendering of the lightweight detection effect of the drone. DETAILED DESCRIPTION

[0073] This embodiment is based on the urban low-altitude logistics scenario, and aims to address the practical problem that delivery drones may encounter illegal interference and threaten the safety of urban logistics airspace, and has constructed a lightweight aerial drone dangerous action detection method. Currently, most urban monitoring systems still rely mainly on manual analysis of visible light videos, which have the limitations of high detection response delay, high false detection rate, and difficulty in real-time identification of high-risk behaviors (such as abnormal hovering, rapid dives, etc.). In order to improve the operational safety and supervision efficiency of the urban low-altitude logistics system, the lightweight aerial drone dangerous action detection method proposed in the present invention is introduced and deployed in the monitoring towers and drone supervision platforms of key urban logistics nodes, realizing real-time detection, trajectory tracking and risk warning of abnormal behaviors of delivery drones, providing technical support for the intelligent upgrade of urban airspace management, combined with Figure 1 , specifically including:

[0074] Step 1: Use a lightweight target detection network to detect video frames Perform drone target positioning, including:

[0075] Step 1.1, feature extraction. Use a lightweight backbone network to extract multi-scale features of the image:

[0076] ;

[0077] in, is the feature of the i-th layer of the network, is a depth-wise separable convolution, It is a point-by-point convolution that achieves lightweight and efficient detection capabilities through the combination of depth-separable convolution and dynamic resolution adjustment.

[0078] Step 1.2, layered attention. Introducing channel attention in feature maps and spatial attention ,get:

[0079] ;

[0080] in, ,Channel attention highlights important channels through global average pooling and weight adjustment, , spatial attention enhances the spatial saliency of the target area through convolution operations.

[0081] Step 1.3, the detection network outputs the target positioning result, such as Figure 2 As shown. The detection box, category and confidence score generated by the lightweight target detection head are:

[0082] ;

[0083] in, is the detection box, which represents the rectangular box of the target location (coordinates of the upper left corner and lower right corner), is the category label, indicating the category to which the target belongs. is the confidence score, which indicates the reliability of the detection result. Is the number of detected targets. Set the confidence threshold , filter low confidence targets:

[0084] .

[0085] Step 2: Combine the detection results and use Kalman filtering to predict and track the target's motion state to solve problems such as detection drift and target loss. Specifically, it includes:

[0086] Step 2.1, each target initializes the state vector as:

[0087] ;

[0088] in are the target center coordinates, is the target width and height, is the velocity component of the central coordinate, is the rate of change of width and height

[0089] Step 2.2, initialize the covariance matrix:

[0090] ;

[0091] Step 2.3, state prediction. Predict the target's next state based on the Kalman filter:

[0092] ;

[0093] in, is the state transition matrix.

[0094] Step 2.4, target association, after the state prediction is completed, the matching score is calculated based on the intersection over union (IoU) of the detection box and the predicted box to associate the detection result with the tracking state:

[0095] ;

[0096] in, Detected object detection box, is the predicted target state. Matching score, which indicates the degree of overlap between the detection box and the prediction box. The matching condition is .in, is the matching IoU threshold. , execute step 3.3 memory recall.

[0097] Step 2.5, state update, update the target state using the detection box through the Kalman filter:

[0098] ;

[0099] in, is the observation value of the detection box, is the observation matrix, which is used to map the state to the observation space, is the Kalman gain, which is calculated as

[0100] ;

[0101] in, is the predicted state covariance matrix, is the measurement noise covariance matrix.

[0102] Step 3: The motion perception memory optimization module uses a dynamic screening mechanism to build an efficient target memory library to enhance tracking capabilities in scenes such as occlusion and rapid motion. Specifically, it includes:

[0103] Step 3.1: Set up the scoring mechanism and calculate the comprehensive score:

[0104] ;

[0105] ;

[0106] ;

[0107] in, is the matching score. is the target score, which indicates the probability of the target existing. is the confidence score of the kth frame, indicating the credibility of the detection result, is the smoothing parameter. In this implementation, , 、 is the weight coefficient used to adjust the impact of different scores.

[0108] Step 3.2: Memory library construction to save high-confidence historical features.

[0109] ;

[0110] in, Features of the i-th target at time k. is the scoring threshold, only memories above this threshold are retained. It is the maximum number of memory frames, used to limit the size of the memory library.

[0111] Step 3.3, memory recall. When , the most recent high-confidence features are retrieved from the memory bank to restore the target state.

[0112] ;

[0113] In this embodiment, Mahalanobis distance is selected for similarity search. Calculate the current prediction state Mahalanobis distance to all states in the memory bank . Set the Mahalanobis distance threshold ,when When , the optimal memory item is selected as , update the target state. > , the target state is not updated.

[0114] Step 4: Analyze motion characteristics and identify dangerous drone actions. This includes:

[0115] Step 4.1, calculate the three-dimensional space coordinates of the target in the camera coordinate system .

[0116]

[0117] Where Z is the depth information, the depth from the target to the camera, obtained by laser ranging. It is the coordinate of the camera's principal point (the intersection of the optical axes) in the image plane, usually located at the center of the image. is the focal length of the camera in pixels.

[0118] Step 4.2, velocity calculation and acceleration calculation.

[0119]

[0120] in, The target is The velocity component in the direction. is the time interval, is the change in position within the time interval.

[0121]

[0122] Step 4.3, flight path angle calculation.

[0123] ;

[0124] ;

[0125] in is the pitch angle, the angle between the drone's motion direction and the horizontal plane, The heading angle is the azimuth of the drone's motion direction on the horizontal plane.

[0126] Step 4.4: Drone anomaly detection. Statistical anomaly detection determines whether a drone exhibits abnormal behavior by analyzing the dynamic change patterns of the feature time series. A feature sequence within a time window T is constructed.

[0127] ;

[0128] Use the dynamic time warping (DTW) algorithm to match the similarity between the current time series and the template sequence. The matching cost of DTW is:

[0129] ;

[0130] in is the time-aligned path, is the kth feature of the current sequence, is the alignment feature of the template sequence.

[0131] Step 4.5: Combine the anomaly scores of multiple features into a total anomaly score:

[0132] ;

[0133] in, Is a weight parameter used to balance the influence of different features. If , it is judged as a dangerous action.

[0134] The present invention can identify abnormal drone behavior by analyzing the drone's flight characteristics in real time, and can generate real-time alarms for dangerous actions (such as rapid descent or approaching sensitive areas) based on the flight characteristics analysis results.

[0135] This embodiment also provides a lightweight aerial drone dangerous action detection system, comprising:

[0136] The target positioning module builds a lightweight target detection network by combining depthwise separable convolution and dynamic resolution adjustment to locate drone targets in captured video frames. It also uses lightweight neural networks to achieve efficient detection of drone targets.

[0137] The motion modeling and tracking module, combined with UAV target positioning, predicts and tracks the target's motion state through Kalman filtering; stable target tracking is achieved through Kalman filtering;

[0138] The motion perception memory optimization module builds a target memory library through a dynamic screening mechanism. During the tracking process, if the target association matching fails, the most recent high-confidence feature is retrieved from the target memory library to update the drone's target status. The memory library is built through a dynamic scoring mechanism to improve the target recovery capability in occluded scenes.

[0139] The motion characteristics analysis module analyzes the motion characteristics of drone targets and detects dangerous drone actions (such as rapid descent and nonlinear motion).

[0140] The innovation of this algorithm lies in its integration of lightweight detection, motion prediction, and motion perception optimization, while focusing on resource efficiency and robustness. This design has obvious advantages in low-resource scenarios, enabling efficient, stable, and adaptable object detection and tracking.

[0141] Obviously, those skilled in the art may make various changes and modifications to the embodiments of the present invention without departing from the spirit and scope of the embodiments of the present invention. Thus, if such changes and modifications of the embodiments of the present invention fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A method for detecting dangerous actions of lightweight aerial drones, characterized in that: include: Step 1: By combining depthwise separable convolution and dynamic resolution adjustment, a lightweight target detection network is constructed to locate the drone target in the collected video frames. Step 2: Combined with the UAV target positioning, the target's motion state is predicted and tracked through Kalman filtering; Step 3: Build a target memory library through a dynamic screening mechanism. When target association matching fails during tracking, retrieve the most recent high-confidence feature from the target memory library to restore the drone's target state. Step 4: Analyze the motion characteristics of the drone target and detect dangerous drone actions; In step 3, a target memory library is constructed through a dynamic screening mechanism, which includes: Calculate the comprehensive score by matching the score and the target score; Based on the comprehensive score, the historical features with high confidence that meet the requirements are saved as the target memory library; The composite score is: ； ； ； in, is the matching score, The detected target detection box, IoU is the intersection over union ratio, is the predicted target state, is the target score, which indicates the probability of the target existing. is the confidence score of the kth frame, is the smoothing parameter, 、 is the weight coefficient; High confidence historical characteristics satisfy: ； in, The features of the i-th target at time k, is the scoring threshold, is the maximum number of memory frames, and t is the frame number.

2. A method for detecting dangerous actions of lightweight aerial drones according to claim 1, characterized in that: The lightweight target detection network includes a feature extraction layer, a hierarchical attention layer, and an output layer. The specific steps of performing drone target positioning on the captured video frames are as follows: Step 1.1: The feature extraction layer uses a lightweight backbone network to extract multi-scale features of the video frame and obtain a feature map. Step 1.2: Hierarchical attention introduces channel attention and spatial attention into the feature map to enhance the feature map; In step 1.3, based on the enhanced feature map, the output layer generates the target detection box, category, and confidence score through the lightweight target detection head. The target with a confidence score exceeding the confidence threshold is regarded as the target detection result.

3. A method for detecting dangerous actions of lightweight aerial drones according to claim 2, characterized in that: The multi-scale features of the video frame extracted in step 1.1 are: ； in, is the feature of the i-th layer of the network, is a depth-wise separable convolution, is point-wise convolution, I t is the video frame; The channel attention and spatial attention in step 1.2 are: ； ； in, is the feature of the i-th layer of the network, AvgPool represents the average pooling operation, W c 、W s Represents weight, Conv represents convolution operation, Activated for Hard-Swish.

4. The method for detecting dangerous actions of a lightweight aerial drone according to claim 1, characterized in that: In step 2, the target's motion state is predicted and tracked through Kalman filtering, which specifically includes: Step 2.1, initialize the state vector of each detected target; Step 2.2, initialize the covariance matrix; Step 2.3: Based on the initialized state vector, the Kalman filter is used to predict the next state of the target. In step 2.4, the matching score is calculated based on the intersection over union (IoU) of the detected state and the predicted state. If the matching score is greater than the IoU threshold, step 2.5 is executed. Otherwise, step 3 is executed to retrieve the most recent high-confidence feature from the target memory and restore the drone's target state. In step 2.5, based on the initialized state vector and covariance matrix, the target state is updated using the detection state through the Kalman filter.

5. A method for detecting dangerous actions of lightweight aerial drones according to claim 4, characterized in that: The next state in step 2.3 is: ； Where F is the state transfer matrix; Step 2.5 updates the target status to: ； ； in, is the observation value of the detection box, is the observation matrix, which is used to map the state to the observation space, is the Kalman gain, is the predicted state covariance matrix, is the measurement noise covariance matrix, The status at the previous moment.

6. The method for detecting dangerous actions of a lightweight aerial drone according to claim 1, characterized in that: The step 4 specifically includes: Step 4.1, calculate the three-dimensional space coordinates of the target in the camera coordinate system; Step 4.2, calculate the velocity and acceleration based on the three-dimensional space coordinates; Step 4.3, calculate the target flight path angle based on the velocity; Step 4.4: Based on the velocity, acceleration, and flight path angle, a feature sequence within a time window is constructed, and the dynamic time warping algorithm is used to match the similarity between the current time feature sequence and the template feature sequence; In step 4.5, based on the similarity, a comprehensive anomaly score is obtained. If the comprehensive anomaly score is greater than the threshold, it is a dangerous action.

7. A method for detecting dangerous actions of lightweight aerial drones according to claim 6, characterized in that: The comprehensive anomaly score in step 4.5 is: ； ； in, is the weight parameter, is the time interval, is the velocity increment, is the acceleration increment, DTW is the dynamic time warping function, is the time-aligned path, is the kth feature of the current sequence, is the alignment feature of the template sequence.

8. A lightweight aerial drone dangerous action detection system implementing the method described in any one of claims 1 to 7, characterized in that: include: The target positioning module builds a lightweight target detection network through the combination of depth-separable convolution and dynamic resolution adjustment to perform drone target positioning on the collected video frames. The motion modeling and tracking module, combined with UAV target positioning, predicts and tracks the target's motion state through Kalman filtering; The motion perception memory optimization module builds a target memory library through a dynamic screening mechanism. During the tracking process, when the target association matching fails, the most recent high-confidence feature is retrieved from the target memory library to restore the drone's target status; The motion characteristics analysis module analyzes the motion characteristics of drone targets and detects dangerous drone actions.

Images