Unmanned aerial vehicle abnormal behavior prediction method based on multi-source data fusion

Abstract

The invention relates to the technical field of unmanned aerial vehicle monitoring, and discloses a multi-source data fusion unmanned aerial vehicle abnormal behavior prediction method, which comprises the following steps: acquiring unmanned aerial vehicle monitoring data acquired by radar, radio and photoelectric monitoring equipment, and generating an unmanned aerial vehicle real-time state vector by adopting a weighted fusion algorithm based on Kalman filtering; inputting the real-time state vector sequence into a long short-term memory network model to predict a future flight trajectory; and comparing the trajectory prediction sequence with the electronic fence boundary data to judge the intrusion risk level and trigger early warning. According to the method, the data credibility is improved through multi-source data fusion, early warning is realized through trajectory prediction, and the problems of low data credibility and response lag in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) monitoring technology, and more specifically, to data fusion technology and trajectory prediction technology, particularly to a method for predicting abnormal behavior of UAVs by fusing multi-source data. Background Technology

[0002] With the rapid development and widespread application of drone technology, drones are increasingly frequently illegally intruding into pre-designated sensitive areas such as airport airspace, military restricted zones, and the vicinity of nuclear facilities, posing a serious threat to airspace safety, public safety, and the security of critical facilities. To address this security challenge, various drone monitoring devices, including radar, photoelectric, and radio monitoring equipment, have been deployed to monitor the flight status of drones in real time.

[0003] Traditional drone monitoring technology primarily relies on single monitoring devices for real-time monitoring. Radar monitoring devices acquire drone position and speed information by emitting electromagnetic waves and receiving reflected signals; photoelectric monitoring devices capture drone flight attitude and appearance features through image recognition technology; and radio monitoring devices obtain device identification information by detecting communication signals between the drone and its remote controller. These monitoring devices can provide certain monitoring capabilities in their respective application scenarios.

[0004] However, existing technologies have two significant drawbacks:

[0005] First, regarding data reliability, the data quality of a single monitoring device is easily affected by environmental factors. While radar monitoring equipment can provide location and velocity information, it is prone to positioning errors in complex electromagnetic environments; photoelectric monitoring equipment experiences a significant drop in accuracy under low visibility weather conditions; and radio monitoring equipment is greatly affected by spectrum interference. The insufficient reliability of a single data source leads to a large number of false alarms or missed alarms in sensitive area intrusion detection, causing difficulties for regulatory work.

[0006] Secondly, regarding response timeliness, existing technologies can only identify anomalies after a drone has entered a sensitive area, lacking the ability to predict the drone's future flight trajectory. This reactive approach means that by the time regulators receive warning information, the drone has already approached or entered the sensitive area, leaving an extremely limited window for interception and intervention, making it difficult to effectively prevent intrusions.

[0007] Therefore, there is a need to provide a method for predicting abnormal behavior of drones that integrates multi-source monitoring data and has trajectory prediction capabilities, in order to solve the technical problems of low data reliability and poor response time in intrusion monitoring in sensitive areas using traditional technologies. Summary of the Invention

[0008] This invention provides a method for predicting abnormal behavior of unmanned aerial vehicles (UAVs) by fusing multi-source data, which solves the technical problems of low reliability of data from single monitoring devices and lack of trajectory prediction capabilities that lead to delayed response in related technologies.

[0009] This invention discloses a method for predicting abnormal behavior of unmanned aerial vehicles (UAVs) based on multi-source data fusion, comprising the following steps: acquiring UAV latitude and longitude coordinates, altitude data, flight speed and heading angle data collected by radar monitoring equipment, communication frequency and equipment identification code collected by radio monitoring equipment, and flight attitude and surrounding environment information collected by photoelectric monitoring equipment; cleaning and normalizing the acquired multi-source monitoring data; fusing the normalized multi-source monitoring data using a weighted fusion algorithm based on Kalman filtering to generate a real-time state vector of the UAV; inputting the UAV real-time state vector sequence within a continuous preset time period into a trained long short-term memory network model to predict the flight trajectory of the UAV in the future time period, generating a trajectory prediction sequence; comparing the trajectory prediction sequence with the electronic fence boundary data of a preset sensitive area to determine the intrusion risk level of the UAV and generate an intrusion risk judgment result; and triggering an intrusion warning based on the intrusion risk judgment result.

[0010] Furthermore, the step of using a weighted fusion algorithm based on Kalman filtering to fuse normalized multi-source monitoring data and generate a real-time UAV state vector includes: establishing measurement noise models for radar monitoring equipment, radio monitoring equipment, and optoelectronic monitoring equipment respectively; calculating the measurement error covariance matrix of each equipment based on historical monitoring data; establishing system state equations and observation equations for the UAV flight state, wherein the system state equation combines the UAV's true state vector at the previous moment with process noise based on the state transition matrix to obtain the state vector at the current moment, and the observation equation combines the true state vector at the current moment with measurement noise based on the observation matrix to obtain the observed values ​​of the monitoring equipment; executing the prediction step of Kalman filtering, calculating the prior state estimate and prior error covariance matrix based on the state transition matrix; executing the update step of Kalman filtering, calculating the Kalman gain for each monitoring equipment, wherein the Kalman gain is calculated based on the prior error covariance matrix, the observation matrix, and the measurement error covariance matrix, and the magnitude of the Kalman gain reflects the degree of contribution of the observed values ​​of the monitoring equipment to the state estimate; and weightedly fusing the observation data of radar, radio, and optoelectronic equipment based on the Kalman gains of each monitoring equipment to generate a real-time UAV state vector.

[0011] Furthermore, before fusing the normalized multi-source monitoring data using a weighted fusion algorithm based on Kalman filtering, the process further includes: dynamically evaluating the data quality of each monitoring device based on the current environmental conditions to generate a reliability coefficient; reducing the reliability coefficient of the radar monitoring device when strong electromagnetic interference is detected; reducing the reliability coefficient of the photoelectric monitoring device when low visibility weather conditions are detected; and introducing the reliability coefficient as an additional weighting factor into the weighted fusion process.

[0012] Further, the step of inputting the real-time state vector sequence of the UAV within a continuously preset time period into the trained Long Short-Term Memory (LSTM) network model to predict the flight trajectory of the UAV in the future time period includes: inputting input data into the input layer of the LTM network model, wherein the input layer receives a state vector sequence of multiple time steps, each state vector containing latitude and longitude coordinates, altitude, flight speed, and heading angle; the data output by the input layer is processed by multiple LTM units, wherein each LTM unit controls the information flow at each time step through the synergistic action of a forget gate, an input gate, and an output gate, wherein the forget gate controls the degree of retention of the cell state of the previous time step, the input gate controls the degree of reception of the current input information, and the output gate controls the degree of output of the cell state to the hidden state; after processing by multiple LTM units, the hidden state vector is obtained at the last time step; the hidden state vector is input into the fully connected output layer, wherein the output layer maps the hidden state to the predicted position coordinates of multiple future time steps through a linear transformation, generating a trajectory prediction sequence.

[0013] Furthermore, the Long Short-Term Memory (LSTM) network model is trained using supervised learning. The training process includes: collecting historical flight data of drones around multiple preset sensitive areas, with each historical data point containing a complete flight trajectory and intrusion label; dividing the historical flight data into training and testing sets; defining a dual objective function, which is a weighted combination of a trajectory prediction error term and an intrusion judgment classification error term. The trajectory prediction error term uses a mean squared error loss function to calculate the deviation between the predicted and actual position coordinates, while the intrusion judgment classification error term uses a cross-entropy loss function to calculate the difference between the predicted intrusion probability and the actual intrusion label; and using an optimization algorithm to iteratively optimize the model parameters to minimize the dual objective function.

[0014] Furthermore, the training set construction process also includes an oversampling step for low, slow, and small UAV samples. This step involves copying or synthesizing the intrusion samples of low, slow, and small UAVs to increase the proportion of such samples in the training set.

[0015] Further, the comparison of the trajectory prediction sequence with the electronic fence boundary data of the preset sensitive area to determine the intrusion risk level of the drone includes: loading the electronic fence boundary data of the preset sensitive area, wherein the electronic fence boundary data defines the restricted area of ​​the sensitive area in the form of three-dimensional spatial coordinates; obtaining the real-time position of the drone at the current moment and determining whether the position falls within the restricted area; if the real-time position falls within the restricted area, the intrusion risk level is determined to be an intrusion; if the real-time position does not fall within the restricted area, all predicted position points in the trajectory prediction sequence are traversed to determine whether any predicted position point falls within the restricted area; if any predicted position point falls within the restricted area, the device identification code is queried to determine whether the drone has access authorization; if no access authorization is granted, the intrusion risk level is determined to be a potential intrusion.

[0016] Furthermore, when the real-time location and all predicted location points do not fall within the restricted area, the method also includes: calculating the shortest distance sequence between each predicted location point in the trajectory prediction sequence and the boundary of the electronic fence; if the distance sequence shows a continuous decreasing trend and the final distance is less than a preset distance threshold, and at the same time, radio monitoring data is queried to confirm that the drone has no access authorization, then the intrusion risk level is determined to be an intrusion risk warning.

[0017] Furthermore, when determining the intrusion risk level of a drone, the method also includes an evaluation step of the confidence level of the predicted trajectory: multiple forward propagations are performed on the same input data while keeping the dropout layer active during the model inference stage to obtain multiple sets of prediction results; the variance of the multiple sets of prediction results is calculated as a measure of prediction uncertainty; when the uncertainty of the predicted location point exceeds a preset uncertainty threshold, the weight of the predicted point in the intrusion risk judgment is reduced.

[0018] This invention also discloses a multi-source data fusion-based UAV abnormal behavior prediction system, comprising: a data acquisition module for acquiring UAV monitoring data collected by radar monitoring equipment, radio monitoring equipment, and photoelectric monitoring equipment; a data fusion module for cleaning and normalizing the multi-source monitoring data, and generating a real-time UAV state vector using a Kalman filter-based weighted fusion algorithm; a trajectory prediction module for inputting the UAV real-time state vector sequence into a long short-term memory network model to generate a trajectory prediction sequence; a risk judgment module for comparing the trajectory prediction sequence with electronic fence boundary data to determine the intrusion risk level; and an early warning triggering module for triggering an intrusion early warning based on the intrusion risk judgment result.

[0019] This invention integrates monitoring data from three types of monitoring devices: radar, radio, and photoelectric. It employs a Kalman filter-based weighted fusion algorithm to dynamically allocate fusion weights according to the measurement noise characteristics of each device. By leveraging the complementarity of different monitoring devices, it enhances data reliability and solves the problem of false alarms and missed alarms caused by insufficient reliability of data from a single monitoring device. Furthermore, it uses a long short-term memory network model to predict the future flight trajectory of drones, assessing intrusion risks before drones enter sensitive areas. This transforms the monitoring mode from traditional real-time monitoring and post-event response to trajectory prediction and early warning, addressing the response lag problem caused by the lack of predictive capabilities in existing technologies. This achieves the technical effects of accurate early warning and early interception.

[0020] Compared to traditional technologies using a single data source, the multi-source data fusion of this invention significantly reduces location judgment errors caused by data deviations from a single device, and reduces false alarms and missed alarms in intrusion detection. Compared to post-event response modes, the trajectory prediction method of this invention enables regulators to obtain early warning information before drones enter sensitive areas, allowing sufficient time windows to execute interception or disposal operations, effectively preventing intrusion and providing effective technical support for the security supervision of sensitive areas. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall process of the multi-source data fusion method for predicting abnormal behavior of unmanned aerial vehicles according to the present invention;

[0022] Figure 2 This is a schematic diagram of the weighted fusion algorithm based on Kalman filtering of the present invention;

[0023] Figure 3 This is a schematic diagram of the trajectory prediction process based on long short-term memory networks according to the present invention;

[0024] Figure 4 This is a schematic diagram of the training process of the long short-term memory network model of the present invention;

[0025] Figure 5 This is a schematic diagram of the intrusion risk level determination process of the present invention;

[0026] Figure 6 This is a schematic diagram of the intrusion warning triggering process of the present invention. Detailed Implementation

[0027] With the rapid development and widespread application of drone technology, drones are increasingly frequently illegally intruding into pre-designated sensitive areas such as airport airspace, military restricted zones, and the vicinity of nuclear facilities, posing a serious threat to airspace security, public safety, and the security of critical facilities. Traditional drone monitoring technology mainly relies on single monitoring devices for real-time monitoring, which has two significant drawbacks:

[0028] First, regarding data reliability, the data quality of a single monitoring device is easily affected by environmental factors. While radar monitoring equipment can provide location and velocity information, it is prone to positioning errors in complex electromagnetic environments; photoelectric monitoring equipment experiences a significant drop in accuracy under low visibility weather conditions (such as rain, fog, and nighttime); and radio monitoring equipment is greatly affected by spectrum interference. The insufficient reliability of a single data source leads to a large number of false alarms or missed alarms in sensitive area intrusion detection.

[0029] Secondly, regarding response timeliness, existing technologies can only identify anomalies after a drone has entered a sensitive area, lacking the ability to predict the drone's future flight trajectory. This "post-event response" model means that by the time regulators receive early warning information, the drone has already approached or entered the sensitive area, leaving an extremely limited window for interception and handling, making it difficult to effectively prevent intrusion.

[0030] Therefore, there is a need to provide a method for predicting abnormal behavior of drones that integrates multi-source monitoring data and has trajectory prediction capabilities, in order to solve the technical problems of low data reliability and poor response time in intrusion monitoring in sensitive areas using traditional technologies.

[0031] This embodiment provides a method for predicting abnormal drone behavior based on multi-source data fusion. It should be understood that abnormal behavior in this embodiment specifically refers to the behavior of drones intruding into a pre-defined sensitive area. This embodiment, based on deployed radar monitoring equipment, radio monitoring equipment, and photoelectric monitoring equipment, achieves early warning of drone intrusion into a pre-defined sensitive area through data fusion and time-series prediction algorithms.

[0032] like Figure 1-6 As shown in the embodiments of this implementation, a method for predicting abnormal drone behavior based on multi-source data fusion includes the following core steps:

[0033] Step 100: Acquire UAV monitoring data collected by multi-source monitoring devices, fuse the multi-source monitoring data, and generate a real-time UAV state vector.

[0034] Step 101: Obtain the latitude and longitude coordinates, altitude data, flight speed and heading angle data of the UAV from the radar monitoring equipment; obtain the communication frequency and equipment identification code between the UAV and the remote controller or base station from the radio monitoring equipment; obtain the flight attitude and surrounding environment information of the UAV from the optoelectronic monitoring equipment; and keep the data collection timestamps of each monitoring equipment synchronized.

[0035] Step 102: Clean the acquired multi-source monitoring data, remove outliers in radar positioning data that exceed the preset jump threshold, remove transient interference values ​​in radio signals that have a duration lower than the preset duration threshold, and use linear interpolation to fill in data gaps caused by equipment obstruction or signal loss, generating a continuous and complete monitoring data sequence.

[0036] The aforementioned linear interpolation method takes the valid data points before and after the missing data points as input, calculates the values ​​of the missing data points through linear interpolation, and outputs the completed continuous monitoring data sequence.

[0037] Step 103: Normalize the cleaned multi-source monitoring data, mapping the values ​​of different dimensions such as height and speed data to the [0,1] interval to eliminate the impact of dimension differences on subsequent data fusion.

[0038] The aforementioned normalization process takes the cleaned multi-source monitoring data as input, applies a minimum-maximum normalization transformation to each data dimension, and outputs normalized monitoring data.

[0039] Step 104: Use a weighted fusion algorithm based on Kalman filtering to fuse the normalized multi-source monitoring data to generate the UAV real-time state vector S. t The vector contains a timestamp t, latitude and longitude coordinates (lat). t ,lon t ), height h t Flight speed v t , heading angle θ t Device Identification Code (ID) t and environmental labels e t .

[0040] It should be noted that the aforementioned weighted fusion algorithm based on Kalman filtering takes the observation data and measurement error covariance matrix of each monitoring device as input, calculates the fusion weight of each device through the prediction and update process of Kalman filtering, and outputs the fused real-time state vector of the UAV. Specifically, step 104 includes the following sub-steps:

[0041] Step 104-1: Establish measurement noise models for radar monitoring equipment, radio monitoring equipment, and photoelectric monitoring equipment respectively, and statistically analyze the measurement error covariance matrix R of each device based on historical monitoring data. radar R radio R optical .

[0042] Step 104-2: Establish the system state equation and observation equation for the UAV's flight state. The system state equation is as follows:

[0043] X t =F t X t-1 +w t

[0044] Among them, X t Let F be the true state vector of the UAV at time t. t Let w be the state transition matrix. tThe noise is a process noise that follows a zero-mean Gaussian distribution.

[0045] The observation equation is:

[0046]

[0047] in, Let be the observation value of the i-th monitoring device at time t. For the observation matrix, To measure the noise, it follows a zero-mean Gaussian distribution, and its covariance matrix is ​​R. i .

[0048] Step 104-3: Perform the prediction step of Kalman filtering and calculate the prior state estimate. and the prior error covariance matrix P t|t-1 :

[0049]

[0050] Among them, Q t Let be the process noise covariance matrix.

[0051] Step 104-4: Perform the Kalman filter update step, calculating the Kalman gain for each monitoring device i. Posterior state estimate and posterior error covariance matrix:

[0052]

[0053] Kalman gain The magnitude of the value reflects the degree of contribution of the observation of the i-th monitoring device to the state estimation, i.e., the fusion weight of that device.

[0054] Step 104-5: Based on the Kalman gain of each monitoring device, perform weighted fusion of the observation data from radar, radio, and optoelectronic devices to generate the final UAV real-time state vector S. t .

[0055] In this embodiment, to further improve the accuracy of data fusion, a device reliability assessment step is included before performing the weighted fusion in step 104. Specifically, the data quality of each monitoring device is dynamically assessed based on the current environmental conditions to generate a reliability coefficient. For example, when strong electromagnetic interference is detected, the reliability coefficient of the radar monitoring device is reduced; when low visibility weather conditions are detected, the reliability coefficient of the photoelectric monitoring device is reduced. The reliability coefficient is introduced as an additional weighting factor into the weighted fusion process in steps 104-5 to further optimize the allocation of fusion weights. This step enables the data fusion process to adaptively adjust the contribution of each data source according to real-time environmental changes, improving the robustness of the fusion result in complex environments.

[0056] Step 200: Input the real-time state vector of the UAV into the trained Long Short-Term Memory network model to predict the flight trajectory of the UAV in the future time period and generate a trajectory prediction sequence.

[0057] Step 201: Obtain the real-time state vector sequence {S} of the UAV within a continuous preset time period. t-T+1 ,S t-T+2 ,…,S t}, where T is the length of the input time window, which constitutes the input data for trajectory prediction.

[0058] Step 202: Input the input data into the input layer of the Long Short-Term Memory Network model. The input layer receives a sequence of state vectors for T time steps. Each state vector contains features such as latitude and longitude coordinates, altitude, flight speed, and heading angle.

[0059] Step 203: The data output from the input layer is processed through multiple layers of Long Short-Term Memory (LSTM) units. The calculation process for each LTM unit at time step τ is as follows:

[0060] The calculation for the forgetting gate is as follows:

[0061] f τ =σ(W f ·[h τ-1 ,x τ ]+b f )

[0062] The input gate is calculated as follows:

[0063] i τ =σ(W i ·[h τ-1 ,x τ 0+b i )

[0064] The candidate cell state is calculated as follows:

[0065]

[0066] The cell state is updated as follows:

[0067]

[0068] The output gate is calculated as follows:

[0069] o τ =σ(W o ·[h τ-1 ,x τ ]+b o )

[0070] The hidden state is calculated as follows:

[0071] h τ =o τ ⊙tanh(C τ )

[0072] Where, x τ h is the input vector at the current time step. τ-1 C is the hidden state from the previous time step. τ-1 Let W be the cell state at the previous time step, and let W and b be the weight matrix and bias vector, respectively. Let σ be the sigmoid activation function, and ⊙ be the element-wise multiplication operation.

[0073] The aforementioned long short-term memory unit controls the degree of retention of the cell state in the previous time step through the forget gate, controls the degree of reception of the current input information through the input gate, and controls the degree of output of the cell state to the hidden state through the output gate, thereby realizing the learning of long-term dependencies in time-series data and capturing flight pattern features such as speed change trends and heading angle change trends.

[0074] Step 204: After processing through multiple layers of long short-term memory units, the hidden state vector h is obtained at the last time step t. t The hidden state vector contains a temporal feature representation of the input sequence.

[0075] Step 205: Transfer the hidden state vector h t The input is fed into a fully connected output layer, which uses a linear transformation to map the hidden state into predicted position coordinates for the next N time steps. The calculation process is as follows:

[0076] Y = W out h t +b out

[0077] Where Y is the output vector, containing the predicted future location sequence {(lat t+1 ,lon t+1 ,h t+1 ),(lat t+2 ,lon t+2 ,h t+2 ),…,(lat t+N ,lon t+N ,h t+N )}, W out Let b be the output layer weight matrix. out The output layer bias vector is used to generate a trajectory prediction sequence.

[0078] It should be noted that the aforementioned Long Short-Term Memory (LSTM) network model is trained using supervised learning. Specifically, the training process includes the following steps:

[0079] Step S1: Collect historical flight data of drones around multiple preset sensitive areas. Each historical data entry contains a complete flight trajectory and an intrusion tag, where the intrusion tag indicates whether the drone eventually entered the preset sensitive area.

[0080] Step S2: Divide the historical flight data into a training set and a test set according to a preset ratio. The training set is used for model parameter optimization, and the test set is used for model performance verification.

[0081] Step S3: Define the dual objective function L total This function is composed of the trajectory prediction error term L. trajectory And the intrusion judgment classification error term L classification composition:

[0082] L total =αL trajectory +βL classification

[0083] Here, α and β are weighting coefficients, satisfying α+β=1.

[0084] The aforementioned trajectory prediction error term L trajectory Using the mean squared error loss function:

[0085]

[0086] in, Let i be the predicted position coordinates at step i. Let be the actual coordinates of the i-th step position.

[0087] The aforementioned intrusion judgment classification error term L classification Using the cross-entropy loss function:

[0088]

[0089] Where M is the number of training samples, y j Let j be the true intrusion label for the j-th sample (0 indicates no intrusion, 1 indicates intrusion). This represents the intrusion probability predicted by the model.

[0090] Step S4: Iteratively optimize the model parameters using the Adam optimization algorithm to minimize the dual objective function L. total Train until the loss function converges or the preset number of training rounds are reached.

[0091] Step S5: Use the test set to verify the performance of the trained model, calculate the average position error of trajectory prediction and the accuracy of intrusion detection, and ensure that the model meets the preset performance indicators.

[0092] In this embodiment of the application, in order to improve the model's ability to detect intrusions by specific types of drones, the training set construction process in step S2 also includes an oversampling step for low-speed, small drone samples.

[0093] The aforementioned oversampling step takes low-altitude, slow-moving, and small UAV intrusion samples from the training set as input. By replicating or synthesizing these samples to enhance their representation in the training set, a balanced training set is output. Low-altitude, slow-moving, and small UAVs possess high stealth capabilities in actual monitoring due to their low flight altitude, slow speed, and small radar cross-section, but their intrusion risk cannot be ignored. This step allows the model to more fully learn the flight characteristics of low-altitude, slow-moving, and small UAVs during training, thereby improving the accuracy of intrusion detection for this type of target and solving the problem of weak model prediction ability for a minority of target types caused by imbalanced samples.

[0094] Step 300: Compare the trajectory prediction sequence with the electronic fence boundary data of the preset sensitive area to determine the intrusion risk level of the drone and generate an intrusion risk assessment result.

[0095] Step 301: Load the electronic fence boundary data of the preset sensitive area. This data defines the restricted area of ​​the sensitive area in the form of three-dimensional spatial coordinates, including latitude and longitude plane boundaries and height restrictions.

[0096] Step 302: Obtain the real-time location of the drone at the current moment (lat t ,lon t ,h t The system determines whether the location falls within the restricted area defined by the electronic fence. If the real-time location falls within the restricted area, the intrusion risk level is determined to be "intruded," and the process proceeds to step 304.

[0097] Step 303: If the real-time position does not fall within the restricted area, then iterate through all predicted position points {(lat)} in the trajectory prediction sequence. t+1 ,lon t+1 ,h t+1 ),…,(lat t+N ,lon t+N ,h t+N The system determines whether any predicted location falls within the restricted area. If at least one predicted location falls within the restricted area, it queries the device identification code in the radio monitoring data to determine whether the drone has access authorization for the sensitive area. If no access authorization is granted, the intrusion risk level is determined to be "potential intrusion"; if access authorization is granted, the intrusion risk level is determined to be "normal flight".

[0098] Step 304: Generate intrusion risk assessment results, which include intrusion risk level, UAV real-time state vector, trajectory prediction sequence, and sensitive area identification.

[0099] It should be noted that the intrusion risk assessment in step 303 above also includes a boundary approach risk assessment method. Specifically, when neither the real-time location nor any of the predicted location points fall within the restricted area, the shortest distance sequence {d} between each predicted location point in the trajectory prediction sequence and the electronic fence boundary is further calculated. t+1 ,d t+2 ,…,d t+N If the distance sequence shows a continuously decreasing trend, i.e., d t+i >d t+i+1 This holds true for all i∈{1,2,…,N-1}, and the final distance d t+N Less than the preset distance threshold D threshold Simultaneously, if radio monitoring data confirms that the drone lacks access authorization, the intrusion risk level is determined to be "Intrusion Risk Warning." This judgment method can identify drones with intrusion tendencies in advance, even before the drone enters a sensitive area and the predicted trajectory does not directly cross the boundary, further improving the foresight of the warning.

[0100] In this embodiment of the application, in order to improve the accuracy of intrusion risk assessment and reduce false alarms, the intrusion risk assessment in step 303 also includes an evaluation step of the confidence level of the predicted trajectory. Specifically, based on the trajectory prediction sequence output by the Long Short-Term Memory network model in step 200, the prediction uncertainty of each predicted location point is calculated simultaneously.

[0101] The aforementioned prediction uncertainty is estimated using the Monte Carlo dropout method. This method takes the trajectory prediction sequence as input, keeps the dropout layer active during the model inference phase, performs multiple forward propagations on the same input data to obtain multiple sets of prediction results, calculates the variance of the prediction results as an uncertainty measure, and outputs the uncertainty value of each predicted location point. When the uncertainty of a predicted location point exceeds a preset uncertainty threshold, the reliability of that predicted point is considered low, and its weight is reduced or it is not used as a basis for intrusion risk assessment. This step avoids false alarms caused by excessive model prediction errors and improves the credibility of intrusion risk assessment.

[0102] It also includes the following steps:

[0103] Step 400: Based on the intrusion risk assessment results, trigger an intrusion warning and output the intrusion warning information.

[0104] Step 401: Determine if the intrusion risk level is any of "Intrusion Already," "Potential Intrusion," or "Intrusion Risk Warning." If so, trigger a high-priority warning; if the intrusion risk level is "Normal Flight," do not trigger a warning.

[0105] Step 402: Generate intrusion warning information, which includes the drone's real-time status vector, a visualized trajectory map of the trajectory prediction sequence, the name of the preset sensitive area and the boundary range of the electronic fence, and the intrusion risk level.

[0106] Step 403: Send the intrusion warning information to the regulatory authority's warning terminal through the warning information sending module, and simultaneously trigger multi-channel notifications, including but not limited to audible and visual alarms, SMS notifications, and mobile application push notifications.

[0107] It should be noted that the multi-channel notification in step 403 above employs a differentiated notification strategy based on the intrusion risk level. Specifically, when the intrusion risk level is "Intrusion Already," the highest priority alert is triggered, all notification channels are activated, and an emergency response work order is automatically generated and assigned to on-duty supervisory personnel. When the intrusion risk level is "Potential Intrusion," the second-highest priority alert is triggered, activating audible and visual alarms and mobile application push notifications. When the intrusion risk level is "Intrusion Risk Warning," only mobile application push notifications are activated to remind supervisory personnel to pay attention to the drone's movements. This differentiated notification strategy ensures the rational allocation of supervisory resources, avoids low-risk events consuming excessive response resources, and ensures that high-risk events are handled promptly.

[0108] This implementation method effectively solves the technical problems of low data reliability and delayed response in traditional UAV monitoring technology by using multi-source data fusion, time-series trajectory prediction and intrusion risk assessment.

[0109] First, addressing the issue of insufficient reliability of data from a single monitoring device, this implementation integrates monitoring data from radar, radio, and optoelectronic devices in step 100. A weighted fusion algorithm based on Kalman filtering is used, dynamically allocating fusion weights according to the measurement noise characteristics of each device. Radar monitoring devices provide high accuracy in position and velocity information under normal electromagnetic conditions but are susceptible to electromagnetic interference; optoelectronic monitoring devices can provide high-resolution image information under good visibility conditions, but their performance degrades under adverse weather conditions; radio monitoring devices provide device identification and communication information and have strong anti-interference capabilities. By fusing data from these three types of devices and leveraging their complementarity, when the data quality of one device deteriorates due to environmental factors, data from other devices can compensate for that device's shortcomings, thereby generating a more reliable real-time UAV state vector. Compared to a single data source, multi-source data fusion significantly reduces position judgment errors caused by deviations in data from a single device, reduces false alarms and false negatives in intrusion detection, and provides a reliable data foundation for subsequent trajectory prediction and intrusion detection.

[0110] Secondly, addressing the issue that traditional technologies can only respond after the fact and cannot provide early warnings, this implementation method employs a Long Short-Term Memory (LSTM) network model in step 200 to predict the future flight trajectory of the drone. The LTM network model, through the synergistic effect of forget gates, input gates, and output gates, can effectively learn long-term dependencies in time-series data, capturing the changing patterns of the drone's flight state in the time dimension, including flight mode characteristics such as speed and heading angle trends. Based on historical and real-time flight state vector sequences, the model can predict the drone's position sequence within a preset time period, forming a complete trajectory prediction sequence. In step 300, by comparing the trajectory prediction sequence with the electronic fence boundary of a preset sensitive area, it is possible to determine in advance whether there is an intrusion risk before the drone enters the sensitive area. This trajectory prediction method transforms the monitoring mode from the traditional "real-time monitoring - after-the-fact response" to "trajectory prediction - early warning," enabling regulators to obtain early warning information before the drone enters the sensitive area, allowing sufficient time windows to execute interception or disposal operations, thereby avoiding actual harm caused by intrusion.

[0111] Furthermore, this implementation method employs a tiered intrusion risk assessment in step 300, classifying intrusion risks into three levels: "already intruded," "potentially intruded," and "intrusion risk warning." If the real-time location falls within the restricted area, it is directly classified as "already intruded," requiring immediate action. If the predicted trajectory crosses the electronic fence boundary, it is classified as "potentially intruded," and an early warning is issued. If the predicted trajectory does not cross the boundary but shows a continuous approaching trend, it is classified as "intrusion risk warning," requiring continuous monitoring. This tiered assessment not only enables timely identification of already intruded behavior but, more importantly, achieves early detection of potential intrusions and risky approaching behaviors through trajectory prediction, significantly improving the foresight of monitoring and early warning.

[0112] In summary, this implementation method improves the reliability of location determination by fusing multi-source data, overcomes the insufficient reliability of data from a single monitoring device, and solves the problem of false alarms and missed alarms caused by low data reliability. By utilizing the temporal trajectory prediction capability of long short-term memory networks, it overcomes the lack of predictive ability in traditional technologies, solves the problem of delayed response and inability to intercept in advance, and achieves accurate and early warning of UAV intrusion into preset sensitive areas, providing effective technical support for the security supervision of sensitive areas.

Claims

1. A method for multi-source data fusion-based abnormal behavior prediction of a UAV, characterized in that, The method comprises the following steps: Obtaining the longitude and latitude coordinates, height data, flight speed and heading angle data of the unmanned aerial vehicle collected by the radar monitoring device, the communication frequency and device identification code collected by the radio monitoring device, and the flight attitude and surrounding environment information collected by the photoelectric monitoring device; cleaning and normalizing the obtained multi-source monitoring data; fusing the normalized multi-source monitoring data by using a weighted fusion algorithm based on Kalman filtering to generate an unmanned aerial vehicle real-time state vector; Inputting the unmanned aerial vehicle real-time state vector sequence in a continuous preset time period into a long short-term memory network model trained to predict the flight trajectory of the unmanned aerial vehicle in a future time period to generate a trajectory prediction sequence; Comparing the trajectory prediction sequence with the electronic fence boundary data of the preset sensitive area to determine the invasion risk level of the unmanned aerial vehicle and generate an invasion risk judgment result; triggering an invasion warning according to the invasion risk judgment result.

2. The method of claim 1, wherein, The method for generating the unmanned aerial vehicle real-time state vector by using the weighted fusion algorithm based on Kalman filtering on the normalized multi-source monitoring data comprises: Establishing a measurement noise model for the radar monitoring device, the radio monitoring device and the photoelectric monitoring device respectively, and calculating the measurement error covariance matrix of each device according to historical monitoring data; Establishing a system state equation and an observation equation of the unmanned aerial vehicle flight state, wherein the system state equation combines the real state vector of the unmanned aerial vehicle at the previous moment and the process noise according to a state transition matrix to obtain the state vector at the current moment, and the observation equation combines the real state vector at the current moment and the measurement noise according to an observation matrix to obtain the observation value of the monitoring device; Performing a prediction step of Kalman filtering to calculate the prior state estimation and the prior error covariance matrix according to the state transition matrix; Performing an update step of Kalman filtering to calculate the Kalman gain for each monitoring device, wherein the Kalman gain is calculated according to the prior error covariance matrix, the observation matrix and the measurement error covariance matrix, and the numerical value of the Kalman gain reflects the contribution degree of the observation value of the monitoring device to the state estimation; Weightedly fusing the observation data of the radar, radio and photoelectric devices according to the Kalman gain of each monitoring device to generate the real-time state vector of the unmanned aerial vehicle.

3. The method of claim 2, wherein, Before the weighted fusion algorithm based on Kalman filtering is used to fuse the normalized multi-source monitoring data, the method further comprises: Dynamically evaluating the data quality of each monitoring device according to the current environmental conditions to generate a reliability coefficient; Lowering the reliability coefficient of the radar monitoring device when strong electromagnetic interference is detected; lowering the reliability coefficient of the photoelectric monitoring device when low-visibility weather conditions are detected; Introducing the reliability coefficient as an additional weight factor into the weighted fusion process.

4. The method of claim 1, wherein, The method for inputting the unmanned aerial vehicle real-time state vector sequence in a continuous preset time period into a long short-term memory network model trained to predict the flight trajectory of the unmanned aerial vehicle in a future time period comprises: Inputting the input data into the input layer of the long short-term memory network model, wherein the input layer receives a state vector sequence at multiple time steps, and each state vector contains longitude and latitude coordinates, height, flight speed and heading angle. The data output by the input layer is processed by multiple long short-term memory units, each of which controls information flow at each time step through the synergistic action of a forgetting gate, an input gate, and an output gate, wherein the forgetting gate controls the degree of retention of the cell state at the previous time step, the input gate controls the degree of reception of the current input information, and the output gate controls the degree of output of the cell state to the hidden state; After processing by multiple long short-term memory units, a hidden state vector is obtained at the last time step; The hidden state vector is input into a fully connected output layer, which maps the hidden state to position coordinate prediction values for multiple future time steps through linear transformation, generating a trajectory prediction sequence.

5. The method of claim 4, wherein, The long short-term memory network model is trained in a supervised learning manner, and the training process includes: Collecting historical flight data of unmanned aerial vehicles around multiple preset sensitive areas, each piece of historical data containing a complete flight trajectory and an intrusion label, and dividing the historical flight data into a training set and a test set; Defining a dual objective function composed of a weighted combination of a trajectory prediction error term and an intrusion judgment classification error term, wherein the trajectory prediction error term calculates the deviation between the predicted position coordinates and the true position coordinates using a mean square error loss function, and the intrusion judgment classification error term calculates the difference between the predicted intrusion probability and the true intrusion label using a cross-entropy loss function; Using an optimization algorithm to iteratively optimize the model parameters to minimize the dual objective function.

6. The method of claim 5, wherein, In the training set construction process, it also includes an oversampling step for low, slow and small unmanned aerial vehicle samples, which increases the proportion of this type of sample in the training set by duplicating or synthesizing low, slow and small unmanned aerial vehicle intrusion samples.

7. The method of claim 1, wherein, The trajectory prediction sequence is compared with the electronic fence boundary data of the preset sensitive area to judge the intrusion risk level of the unmanned aerial vehicle, including: Loading the electronic fence boundary data of the preset sensitive area, which defines the no-entry range of the sensitive area in the form of three-dimensional space coordinates; Obtaining the real-time position of the unmanned aerial vehicle at the current time, and determining whether the position falls within the no-entry range; if the real-time position falls within the no-entry range, the intrusion risk level is determined to be invaded; If the real-time position does not fall within the no-entry range, all predicted position points in the trajectory prediction sequence are traversed to determine whether any predicted position point falls within the no-entry range; if there is a predicted position point that falls within the no-entry range, the equipment identification code is queried to determine whether the unmanned aerial vehicle has access authorization; if there is no access authorization, the intrusion risk level is determined to be a potential intrusion.

8. The method of claim 7, wherein, When neither the real-time position nor all predicted position points fall within the no-entry range, it also includes: Calculating the shortest distance sequence between each predicted position point in the trajectory prediction sequence and the electronic fence boundary; If the distance sequence shows a continuous decreasing trend and the final distance is less than a preset distance threshold, and the radio monitoring data confirms that the unmanned aerial vehicle has no access authorization, the intrusion risk level is determined to be an intrusion risk warning.

9. The method of claim 7, wherein, When judging the intrusion risk level of the unmanned aerial vehicle, it also includes an evaluation step of the prediction trajectory confidence: Multiple forward propagations are performed on the same input data in the model inference stage with dropout layer activation, obtaining multiple sets of prediction results; a variance of the plurality of sets of prediction results is calculated as a prediction uncertainty measure; when the uncertainty of the predicted position point exceeds a preset uncertainty threshold, the weight of the predicted point in the intrusion risk judgment is reduced.

10. A multi-source data fusion unmanned aerial vehicle abnormal behavior prediction system for performing the method of any one of claims 1 to 9, characterized in that, comprise: a data acquisition module configured to acquire unmanned aerial vehicle monitoring data collected by a radar monitoring device, a radio monitoring device, and an optoelectronic monitoring device; a data fusion module configured to clean and normalize multi-source monitoring data, and generate an unmanned aerial vehicle real-time state vector by using a weighted fusion algorithm based on Kalman filtering; a trajectory prediction module configured to input the unmanned aerial vehicle real-time state vector sequence into a long short-term memory network model to generate a trajectory prediction sequence; a risk judgment module configured to compare the trajectory prediction sequence with electronic fence boundary data to determine an intrusion risk level; an early warning triggering module configured to trigger an intrusion early warning according to the intrusion risk judgment result.

Images