An unmanned aerial vehicle trajectory tracking system based on multi-sensor fusion

Abstract

The application discloses a kind of unmanned aerial vehicle trajectory tracking systems based on multi-sensor fusion, it is related to unmanned system autonomous navigation and control technical field, including unmanned aerial vehicle data acquisition module, unmanned aerial vehicle feature extraction module, trajectory prediction deviation score module, trajectory dynamic correction module and trajectory monitoring module;Unmanned aerial vehicle data acquisition module acquires and pre-processes unmanned aerial vehicle and GPS data, unmanned aerial vehicle feature extraction module constructs trajectory tracking feature sequence list, trajectory prediction deviation score module outputs three-dimensional trajectory prediction coordinates and deviation degree score, trajectory dynamic correction module generates correction parameter, based on determination logic conversion unmanned aerial vehicle correction instruction and attitude feedback, trajectory monitoring module constructs visual interface.The application significantly improves the accuracy of unmanned aerial vehicle trajectory tracking in complex environment, realizes real-time detection and dynamic correction of abnormal deviation, three-dimensional visual interface enhances state monitoring capability, reduces trajectory tracking error, reduces misjudgment rate, ensures flight safety and control efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of autonomous navigation and control of unmanned systems, and particularly relates to a multi-sensor fusion-based unmanned aerial vehicle trajectory tracking system. BACKGROUND

[0002] Unmanned aerial vehicle trajectory tracking refers to a process of real-time monitoring, recording and analyzing the flight path of an unmanned aerial vehicle in three-dimensional space using advanced technical means. This process usually combines global positioning systems, inertial navigation systems, radars and visual recognition technologies to obtain accurate position, speed, height and attitude information of the unmanned aerial vehicle. Through processing and analysis of these data, the flight trajectory of the unmanned aerial vehicle can be constructed, and thus comprehensive monitoring and management of the flight state of the unmanned aerial vehicle can be realized. Trajectory tracking not only helps to ensure the safe flight of the unmanned aerial vehicle along the predetermined route, but also enables rapid response in emergency situations, such as obstacle avoidance and return, thereby providing important protection for the safe flight of the unmanned aerial vehicle.

[0003] In order to solve the problems of insufficient trajectory prediction accuracy and high false positive rate of abnormal deviation detection in complex environments for unmanned aerial vehicle trajectory tracking, the existing technology uses a single sensor data-driven and fixed threshold rule detection method for processing. However, when the GPS signal is blocked and subject to electromagnetic interference, the trajectory prediction error increases sharply, and the inertial data high-frequency noise triggers false positives, which in turn leads to trajectory tracking errors, long response delay time for abnormal deviation, and frequent false triggering of correction instructions causing flight oscillation. In order to solve these problems, a multi-sensor fusion-based unmanned aerial vehicle trajectory tracking system is proposed. SUMMARY

[0004] The present application aims to provide a multi-sensor fusion-based unmanned aerial vehicle trajectory tracking system to solve the problems raised in the background.

[0005] To solve the above technical problems, the technical solution adopted by the present application is as follows: a multi-sensor fusion-based unmanned aerial vehicle trajectory tracking system, comprising an unmanned aerial vehicle data acquisition module, an unmanned aerial vehicle feature extraction module, a trajectory prediction deviation scoring module, a trajectory dynamic correction module and a trajectory monitoring module.

[0006] The unmanned aerial vehicle data acquisition module acquires and pre-processes unmanned aerial vehicle data and GPS data.

[0007] The unmanned aerial vehicle feature extraction module extracts features from the pre-processed unmanned aerial vehicle data and GPS data, and constructs a trajectory tracking feature sequence list.

[0008] The trajectory prediction deviation scoring module combines the trajectory tracking feature sequence list to construct a trajectory prediction model and an anomaly detection model, and outputs three-dimensional trajectory prediction coordinates and trajectory deviation score.

[0009] The trajectory dynamic correction module dynamically weights the trajectory prediction coordinates and the trajectory deviation degree score to obtain a trajectory correction parameter of the unmanned aerial vehicle, constructs a judgment logic based on the trajectory deviation degree score, converts the trajectory correction parameter into an unmanned aerial vehicle correction instruction, and generates a posture adjustment feedback quantity.

[0010] The trajectory monitoring module constructs a visual interface containing a predicted trajectory line, an actual trajectory line and a safety envelope surface according to the posture adjustment feedback quantity.

[0011] The further improvement of the technical scheme of the application is that in the unmanned aerial vehicle data acquisition module, the acquisition and preprocessing process of the unmanned aerial vehicle data and the GPS data includes:

[0012] The unmanned aerial vehicle data includes unmanned aerial vehicle three-axis acceleration and angular velocity data, and the GPS data includes GPS latitude and longitude coordinates and velocity scalar data.

[0013] The inertial measurement unit is fixed on the gravity center plane of the unmanned aerial vehicle, adopts a three-axis orthogonal layout, and is aligned with the body coordinate system, the inertial measurement unit measures three-axis acceleration data in the body coordinate system through a three-axis accelerometer, and measures angular velocity data around the body coordinate axis through a three-axis gyroscope, the collected unmanned aerial vehicle three-axis acceleration and angular velocity data are cached to a ring queue with a queue length of 50 groups according to time stamps;

[0014] The global positioning system receiver is arranged on the top of the unmanned aerial vehicle, the antenna is directed to an unobstructed area of the sky, the global positioning system receiver receives satellite signals to solve latitude and longitude coordinates and velocity scalar data, and the collected latitude and longitude coordinates and velocity scalar data are stored in a first-in-first-out buffer with a buffer capacity of 20 groups after being parsed according to a protocol;

[0015] The inertial measurement unit is directly connected to the main control module of the unmanned aerial vehicle trajectory tracking system through a four-wire serial bus, the sampling frequency is set to 200 Hz, a shock-absorbing base is additionally provided, the global positioning system receiver adopts an asynchronous serial communication protocol, the sampling frequency is 10 Hz, is synchronized with the clock signal of the inertial measurement unit, and a built-in backup power supply is provided.

[0016] High-frequency noise of the unmanned aerial vehicle three-axis acceleration and angular velocity data is suppressed through Kalman filtering technology, a state transition matrix and an observation noise covariance matrix are designed, true values of the acceleration and the angular velocity are iteratively predicted and updated, the filtered data are synchronized and aligned according to time stamps after filtering, and then cached to the ring queue, and the three-axis acceleration and angular velocity data after noise suppression are output;

[0017] The sliding window mean filtering is adopted, the latitude and longitude coordinates and the velocity scalar in the window are respectively calculated, the data points with the direction difference of the velocity vector exceeding 30 degrees are removed and replaced by the median in the window, meanwhile, the buffer interpolation algorithm is enabled in the signal interruption period, the missing values are linearly extrapolated by using the historical data, and the smoothed latitude and longitude coordinates and velocity scalar data are output.

[0018] Further improvement of the technical scheme of the application lies in that the feature extraction process of the preprocessed UAV data in the UAV feature extraction module comprises:

[0019] The time domain variance features of the acceleration and angular velocity data of each axis in the preprocessed window are obtained, and three-axis acceleration variance and three-axis angular velocity variance are obtained.

[0020] The three-axis acceleration and angular velocity data of the same window are respectively subjected to fast Fourier transform, the frequency domain amplitude spectrum is obtained, the three frequency points with the maximum amplitude in the frequency domain are extracted, the proportion of the energy of the three frequency points to the total energy of all frequency points in the Nyquist frequency is analyzed, and the frequency domain principal component energy proportion feature of the acceleration and angular velocity is output.

[0021] Further improvement of the technical scheme of the application lies in that the feature extraction process of the preprocessed GPS data in the UAV feature extraction module comprises:

[0022] Based on the latitude and longitude coordinates of the three consecutive points in the preprocessed sliding window, the latitude and longitude coordinates are converted into plane coordinates, the reciprocal of the curvature radius of the arc formed by the three points is analyzed, and the trajectory curvature feature of the current window is output.

[0023] Based on the preprocessed velocity scalar data, the heading angle of the sampling point is analyzed according to the difference between the adjacent latitude and longitude coordinates, the length change of the velocity vector of the adjacent sampling points is obtained, and the velocity change rate feature is obtained.

[0024] The time domain variance, the frequency domain principal component energy proportion, the trajectory curvature and the velocity change rate are aligned by time stamp interpolation, and are integrated into the trajectory tracking feature sequence list.

[0025] Further improvement of the technical scheme of the application lies in that in the trajectory prediction deviation scoring module, the trajectory prediction model is constructed in combination with the trajectory tracking feature sequence list, and the process of outputting the three-dimensional trajectory prediction coordinates comprises:

[0026] A stacked double memory unit layer based on a long short-term memory network architecture is adopted, the first layer and the second layer each contain 32 memory units, 10 groups of data are taken as a time window, the time domain variance feature and the velocity change rate feature are combined into a 7-dimensional prediction sequence, each feature is respectively subjected to maximum-minimum normalization, and the trajectory prediction model is constructed.

[0027] The predicted sequence is input into the Long Short-Term Memory Network model step by step. The information flow is controlled by the internal forget gate and input gate. The memory unit stores the long-term dependencies between temporal features. The hidden state output at the end of the output gate is mapped to three-dimensional trajectory prediction coordinates through a fully connected layer. The three-dimensional trajectory prediction coordinates include the longitude coordinates, latitude coordinates and altitude coordinates of the UAV.

[0028] Mean squared error is used to measure the difference between the predicted coordinates and the actual coordinates of the 3D trajectory. The partial derivatives of the loss function with respect to the weights and biases of each layer are calculated using the gradient descent method. The maximum number of iterations is set to 5000, and training is terminated when the maximum number of iterations is reached.

[0029] A further improvement to the technical solution of this invention lies in the following: In the trajectory prediction deviation scoring module, the process of constructing an anomaly detection model by combining the trajectory tracking feature sequence list and outputting the trajectory deviation score includes:

[0030] Align the energy proportion features and trajectory curvature features of the principal components in the frequency domain by timestamp to construct a single-sample input vector, and perform zero-mean standardization on each feature dimension;

[0031] An anomaly detection model is constructed using the Isolation Forest algorithm. 80% of historical data is randomly selected as the training set, and 20% as the validation set. Isolated trees are trained independently, with 128 samples randomly selected from each isolated tree as a training subset. The maximum depth is set to 8. Split points are selected for each feature dimension, recursively splitting the data until isolated samples are reached and the depth limit is exceeded, generating 100 isolated trees to form an isolated forest. Based on a single sample input vector, its path length in each isolated tree is obtained. The path lengths of the constructed isolated trees and the baseline path length are then combined. Calculate trajectory deviation score The calculation process is as follows:

[0032] ;

[0033] ;

[0034] ;

[0035] in, In isolated trees The mean, The total number of training samples, For harmonic numbers, ;

[0036] Based on the statistical deviation score distribution of the validation set, the mean and standard deviation are calculated, and an outlier threshold is set. The trajectory deviation is determined when the trajectory deviation score is greater than the abnormal threshold, and the path length reference value is updated every 24 hours using new data to retrain 10 isolated trees.

[0037] The trajectory dynamic correction module dynamically weights the trajectory prediction coordinates and the trajectory deviation score to obtain the trajectory correction parameters of the unmanned aerial vehicle.

[0038] The prediction coordinate weight and the deviation compensation weight are dynamically set according to the ratio of the trajectory deviation score to the abnormal threshold , and the sum of the prediction coordinate weight and the deviation compensation weight is 1, and the calculation process is as follows:

[0039] ;

[0040] ;

[0041] wherein, is an adjustment factor;

[0042] When , the direction compensation amount and the height compensation amount are calculated based on the trajectory curvature feature and the speed change rate feature , so as to obtain the trajectory correction parameters of the unmanned aerial vehicle, the trajectory correction parameters including the direction correction amount , the speed correction amount and the height correction amount , and the calculation process is as follows:

[0043] ;

[0044] ;

[0045] ;

[0046] ;

[0047] ;

[0048] wherein, is a curvature-speed coupling coefficient, is a safety envelope height, is a height tolerance coefficient, is a predicted height, is a predicted heading angle, is a predicted speed, is a speed correction gain.​

[0049] The further improvement of the technical scheme of the present application is that in the trajectory dynamic correction module, the process of converting the trajectory correction parameter into the UAV correction instruction based on the trajectory deviation degree score and the determination logic comprises:

[0050] When the deviation occurs for three times in succession , it is determined that the deviation is continuous, and the correction instruction is triggered; when the deviation does not occur for more than three times in succession , only the log is recorded, and the control instruction is not triggered, wherein the correction instruction comprises a direction correction instruction , a speed correction instruction and a height correction instruction , and the calculation process is as follows:

[0051] ;

[0052] ;

[0053] ;

[0054] ;

[0055] wherein, K is a proportional coefficient, K I is an integral coefficient, is the deviation of the heading angle in the last three times, is the speed deviation, is the current actual speed, is a speed tracking coefficient, is a height tracking coefficient. The further improvement of the technical scheme of the present application is that in the trajectory dynamic correction module, the process of generating the attitude adjustment feedback quantity comprises:

[0056] The proportional-integral-derivative controller is used to obtain the heading angle error, the speed error and the height error in real time based on the difference between the correction quantity

[0057] , and and the corresponding actual measured values, and the proportional term, the integral term and the differential term are dynamically adjusted to control the weight, the differential term suppresses high-frequency oscillation, the integral term eliminates steady-state deviation, and the attitude adjustment feedback quantity is output, wherein the attitude adjustment feedback quantity comprises the rudder adjustment quantity, the throttle adjustment quantity and the elevator adjustment quantity. The further improvement of the technical scheme of the present application is that in the trajectory monitoring module, according to the attitude adjustment feedback quantity, the process of constructing the visual interface comprising the predicted trajectory line, the actual trajectory line and the safety envelope surface comprises:

[0058]

[0059] ​The trajectory prediction coordinates output by the integrated trajectory prediction model, the smoothed trajectory of the global positioning system and the safety envelope surface parameters are integrated, plane projection conversion is used to map the longitude and latitude coordinates into three-dimensional space coordinates, a red predicted trajectory line, a blue actual trajectory line and a semi-transparent green columnar safety envelope surface are constructed, the interface is refreshed at a period of 0.1 seconds, real-time attitude adjustment feedback value numbers are synchronously displayed, zooming, rotating and trajectory characteristic parameter browsing functions are supported;

[0060] When the actual trajectory point continuously exceeds the safety envelope surface, a secondary warning mechanism is triggered, the primary warning mark deviates from the point and a pop-up window is prompted, the secondary warning forcibly switches the control mode, the operator can manually adjust the envelope surface tolerance parameters, and the time domain variance and the frequency domain principal component energy proportion are viewed when the trajectory line is clicked.

[0061] Thanks to the above technical solutions, the present application has the following technical progress compared with the prior art:

[0062] 1. The unmanned aerial vehicle trajectory tracking system based on multi-sensor fusion provided by the present application significantly improves the trajectory prediction accuracy in complex electromagnetic interference and GPS signal shielding scenarios by combining the spatiotemporal alignment data fusion of the inertial measurement unit and the global positioning system with the time series modeling capability of the long short-term memory network, and reduces the measured prediction error compared with the traditional single-sensor scheme.

[0063] 2. The unmanned aerial vehicle trajectory tracking system based on multi-sensor fusion provided by the present application uses the isolated forest algorithm to analyze the cross-modal features of the frequency energy proportion and the trajectory curvature, and combines the continuous threshold crossing judgment logic of the deviation degree score, so that the abnormal detection false alarm rate is lower than that of the traditional fixed threshold method, the response speed is improved, and flight oscillation caused by false triggering of correction instructions is effectively avoided.

[0064] 3. The unmanned aerial vehicle trajectory tracking system based on multi-sensor fusion provided by the present application generates direction, speed and height correction parameters through a dynamic weighted fusion layer, and constructs a three-dimensional geographic information visualization interface to real-time map the predicted trajectory, the actual trajectory and the safety envelope surface, realizes full-dimensional monitoring of the trajectory state and a human intervention channel, improves the decision-making efficiency of the operator, and prolongs the system safety fault tolerance time. BRIEF DESCRIPTION OF DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0066] Figure 1 The block diagram of the present application. DETAILED DESCRIPTION

[0067] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] Examples, such as Figure 1 As shown, the present invention provides a drone trajectory tracking system based on multi-sensor fusion, including a drone data acquisition module, a drone feature extraction module, a trajectory prediction deviation scoring module, a trajectory dynamic correction module, and a trajectory monitoring module;

[0069] The UAV data acquisition module collects and preprocesses UAV and GPS data. UAV data includes three-axis acceleration and angular velocity data, while GPS data includes GPS latitude and longitude coordinates and velocity scalar data. The inertial measurement unit (IMU) is fixed to the UAV's center of gravity plane, using a three-axis orthogonal layout aligned with the body coordinate system. The IMU measures three-axis acceleration data in the body coordinate system using a three-axis accelerometer and angular velocity data around the body coordinate axes using a three-axis gyroscope. The collected UAV three-axis acceleration and angular velocity data are cached in a circular queue with 50 groups, timestamped. The GPS receiver is positioned on top of the UAV with its antenna facing an unobstructed area. The GPS receiver receives satellite signals and calculates latitude and longitude coordinates and velocity scalar data. The collected latitude and longitude coordinates and velocity scalar data are parsed according to the protocol and stored in a first-in-first-out (FIFO) buffer with a capacity of 20 groups. The IMU communicates with the UAV via a four-wire serial bus. The main control module of the trajectory tracking system is directly connected, with a sampling frequency of 200 Hz. A shock-absorbing base is added. The GPS receiver adopts an asynchronous serial communication protocol with a sampling frequency of 10 Hz, synchronized with the clock signal of the inertial measurement unit. It has a built-in backup power supply. Kalman filtering technology is used to suppress high-frequency noise in the three-axis acceleration and angular velocity data of the UAV. A state transition matrix and observation noise covariance matrix are designed to iteratively predict and update the true values ​​of acceleration and angular velocity. The filtered data is cached in a circular queue after being aligned by timestamp. The noise-suppressed three-axis acceleration and angular velocity data are output. A sliding window mean filtering is used to calculate the arithmetic mean of latitude and longitude coordinates and velocity scalars within the window. Data points with an outlier difference of more than 30 degrees between adjacent points of velocity vector are removed and replaced with the median within the window. At the same time, a buffer interpolation algorithm is used during signal interruption periods to linearly extrapolate missing values ​​using historical data and output smoothed latitude and longitude coordinates and velocity scalar data.

[0070] The UAV feature extraction module extracts features from the preprocessed UAV data and GPS data, constructs a trajectory tracking feature sequence list, obtains the time domain variance features of the acceleration and angular velocity data of each axis in the window after preprocessing, obtains the three-axis acceleration variance and three-axis angular velocity variance, respectively performs fast Fourier transform on the three-axis acceleration and angular velocity data of the same window, obtains the frequency domain amplitude spectrum, extracts the three frequency points with the largest amplitude, analyzes the proportion of the energy of the three frequency points in the total energy of all frequency points within the Nyquist frequency, outputs the frequency domain principal component energy proportion features of acceleration and angular velocity, converts the longitude and latitude coordinates of the three consecutive points in the sliding window after preprocessing into plane coordinates, analyzes the reciprocal of the curvature radius of the circular arc formed by the three points, and outputs the trajectory curvature feature of the current window. Based on the preprocessed speed scalar data, the heading angle of the sampling point is analyzed according to the difference between adjacent longitude and latitude coordinates, the length change of the velocity vector of adjacent sampling points is obtained, and the speed change rate feature is obtained. The time domain variance, frequency domain principal component energy proportion, trajectory curvature and speed change rate are aligned by time stamp interpolation, and are integrated into a trajectory tracking feature sequence list.

[0071] The trajectory prediction deviation scoring module combines the trajectory tracking feature sequence list to construct a trajectory prediction model and an anomaly detection model, and outputs three-dimensional trajectory prediction coordinates and trajectory deviation score. A stacked double memory unit layer based on a long short-term memory network architecture is adopted, the first layer and the second layer each include 32 memory units, 10 groups of data are taken as a time window, the time domain variance features and the speed change rate features are combined into a 7-dimensional prediction sequence, and each feature is normalized by maximum-minimum respectively. The trajectory prediction model is constructed, the prediction sequence is input into the long short-term memory network model in time steps, the information flow is controlled through the internal forgetting gate and the input gate, the memory unit stores the long-term dependence relationship between time sequence features, and the hidden state output at the last time step is mapped into three-dimensional trajectory prediction coordinates through a full connection layer. The three-dimensional trajectory prediction coordinates include longitude coordinates, latitude coordinates and height coordinates. The mean square error is used to measure the difference between the three-dimensional trajectory prediction coordinates and the real trajectory coordinates, the partial derivatives of the loss function with respect to the weights and biases of each layer are calculated by the gradient descent method, the maximum iteration number is set to 5000 times, and the training is terminated when the maximum iteration number is reached. The frequency domain principal component energy proportion features and the trajectory curvature features are aligned by time stamp, a single sample input vector is constructed, and each feature is zero-mean standardized. An anomaly detection model is constructed using the isolated forest algorithm, 80% of the historical data are randomly selected as the training set and 20% as the validation set, and isolated trees are independently trained. Each isolated tree randomly selects 128 groups of samples as a training subset, the maximum depth is set to 8, and a split point is selected for each feature dimension. The data is recursively divided until the samples are isolated and the depth limit is reached, 100 isolated trees are generated to form an isolated forest. Based on the single sample input vector, the path length in each isolated tree is obtained, and the path length of the constructed isolated tree and the path length reference value are integrated Calculate trajectory deviation score The calculation process is as follows:

[0072] ;

[0073] ;

[0074] ;

[0075] in, In isolated trees The mean, The total number of training samples, For harmonic numbers, Based on the statistical deviation score distribution of the validation set, the mean and standard deviation are calculated, and an outlier threshold is set. The mean is the sum of three times the standard deviation. When the trajectory deviation score is greater than the anomaly threshold, it is judged as trajectory deviation. Ten isolated trees are retrained with new data every 24 hours, and the baseline value of path length is updated.

[0076] The trajectory dynamic correction module dynamically weights the predicted trajectory coordinates and the trajectory deviation score to obtain the UAV's trajectory correction parameters. Based on the trajectory deviation score, it constructs a judgment logic, converts the trajectory correction parameters into UAV correction commands, and generates attitude adjustment feedback. The module dynamically sets the predicted coordinate weights according to the ratio of the trajectory deviation score to the anomaly threshold. and deviation compensation weight And the sum of the predicted coordinate weights and the deviation compensation weights is equal to 1. The calculation process is as follows:

[0077] ;

[0078] ;

[0079] in, As a regulating factor, when Based on trajectory curvature features Characteristics of velocity change Calculate directional compensation amount With height compensation Thus, the trajectory correction parameters of the UAV are obtained, including the direction correction amount. Speed ​​correction amount and height correction amount The calculation process is as follows:

[0080] ;

[0081] ;

[0082] ;

[0083] ;

[0084] ;

[0085] wherein, is a curvature-velocity coupling coefficient, is a safety envelope height, is a height tolerance coefficient, is a predicted height, is a predicted heading angle, is a predicted velocity, is a velocity correction gain, when the deviation is continuously three times , it is determined that the deviation is continuous, and a correction instruction is triggered, when the deviation is continuously not more than three times , only a log is recorded, and no control instruction is triggered, the correction instruction includes a direction correction instruction , a velocity correction instruction and a height correction instruction , the calculation process of which is as follows:

[0086] ;

[0087] ;

[0088] ;

[0089] ;

[0090] wherein, is a proportional coefficient, is an integral coefficient, is a deviation of the heading angle in the last three times, is a velocity deviation, is a current actual velocity, is a velocity tracking coefficient, is a height tracking coefficient, through a proportional-integral-derivative controller, based on the difference between the correction amount , and and the corresponding actual measured value, the heading angle error, the velocity error and the height error are obtained in real time, the control weight is dynamically adjusted in combination with the proportional term, the integral term and the differential term, the differential term suppresses high-frequency oscillation, and the integral term eliminates steady-state deviation, and an attitude adjustment feedback amount is output, the attitude adjustment feedback amount includes a rudder adjustment amount, a throttle adjustment amount and an elevator adjustment amount;

[0091] The trajectory monitoring module constructs a visual interface including a predicted trajectory line, an actual trajectory line and a safety envelope surface according to the attitude adjustment feedback, integrates the trajectory prediction coordinates output by the trajectory prediction model, the smoothed trajectory of the global positioning system and the safety envelope surface parameters, maps the latitude and longitude coordinates to three-dimensional space coordinates by using plane projection conversion, constructs a red predicted trajectory line, a blue actual trajectory line and a semi-transparent green columnar safety envelope surface, the interface is refreshed at a period of 0.1 seconds, the real-time attitude adjustment feedback value is displayed synchronously, zooming, rotating and trajectory feature parameter browsing functions are supported, when the actual trajectory point continuously exceeds the safety envelope surface, a secondary warning mechanism is triggered, the first warning marks the deviation point and pops up a prompt, the secondary warning forcibly switches the control mode, the operator can manually adjust the envelope surface tolerance parameters, and the time domain variance and frequency domain principal component energy proportion are viewed by clicking the trajectory line.

[0092] Firstly, the three-axis acceleration and angular velocity data of the inertial measurement unit and the latitude, longitude and speed data of the global positioning system are acquired and preprocessed in real time by the unmanned aerial vehicle data acquisition module, then the time domain variance, frequency domain principal component energy proportion, trajectory curvature and speed change rate features are extracted by the unmanned aerial vehicle feature extraction module, a time-aligned trajectory tracking feature sequence list is constructed, next, the trajectory prediction deviation scoring module synchronously runs the long short-term memory network trajectory prediction model and the isolation forest anomaly detection model based on the feature sequence list, and outputs the three-dimensional prediction coordinates in the future 0.5 seconds and the real-time deviation scoring respectively, on this basis, the trajectory dynamic correction module dynamically allocates the weight coefficients of the prediction coordinates and the deviation compensation by using the Sigmoid function, when the deviation scoring continuously exceeds the threshold value for three times, the generated heading, speed and height correction parameters are converted into proportional-integral control instructions, finally, the trajectory monitoring module maps the correction instructions to the attitude feedback, combines the predicted trajectory line, the actual trajectory line and the dynamic safety envelope surface, and realizes the full-dimensional visual monitoring of the flight state in the three-dimensional geographic information interface.

[0093] The above merely describes specific embodiments of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A multi-sensor fusion based unmanned aerial vehicle (UAV) trajectory tracking system, characterized in that: The unmanned aerial vehicle data acquisition module, the unmanned aerial vehicle feature extraction module, the trajectory prediction deviation scoring module, the trajectory dynamic correction module and the trajectory monitoring module are included. The unmanned aerial vehicle data acquisition module acquires and pre-processes unmanned aerial vehicle data and GPS data; The unmanned aerial vehicle feature extraction module extracts features from the pre-processed unmanned aerial vehicle data and GPS data, and constructs a trajectory tracking feature sequence list; The trajectory prediction deviation scoring module combines the trajectory tracking feature sequence list to construct a trajectory prediction model and an anomaly detection model, and outputs three-dimensional trajectory prediction coordinates and trajectory deviation scores. The frequency domain principal component energy proportion feature and the trajectory curvature feature are aligned by timestamp, a single sample input vector is constructed, and each dimension of the feature is zero-mean standardized. An isolated forest algorithm is used to construct an anomaly detection model, 80% of historical data is randomly selected as a training set, 20% is randomly selected as a verification set, an isolated tree is independently trained, 128 groups of samples are randomly selected as a training subset for each isolated tree, the maximum depth is set to 8, a split point is selected for each feature dimension, the data is recursively divided until the samples are isolated and the depth limit is reached, 100 isolated trees are generated to constitute an isolated forest, based on a single sample input vector, the path length in each isolated tree is obtained, and the path length of the constructed isolated tree and the path length reference value are integrated , the trajectory deviation degree score is calculated , and the calculation process is as follows: ； ； ； wherein, is the mean of the isolated trees , is the total number of training samples, is the harmonic number, ; Based on the statistical deviation score distribution of the verification set, the mean and standard deviation are calculated, and the abnormal threshold is set The sum of the mean and 3 times the standard deviation, when the trajectory deviation score is greater than the abnormal threshold, is determined as the trajectory deviation, and the path length reference value is updated every 24 hours with new data to retrain 10 isolated trees. The trajectory dynamic correction module dynamically weights the trajectory prediction coordinates and the trajectory deviation scores to obtain the trajectory correction parameters of the unmanned aerial vehicle, constructs a judgment logic based on the trajectory deviation scores, converts the trajectory correction parameters into unmanned aerial vehicle correction instructions, and generates attitude adjustment feedback, thereby obtaining the trajectory correction parameters of the unmanned aerial vehicle. According to the ratio of the trajectory deviation score and the abnormal threshold, the prediction coordinate weight is dynamically set and the deviation compensation weight , and the sum of the prediction coordinate weight and the deviation compensation weight is 1, and the calculation process is as follows: ； ； wherein is a modulating factor; When , based on the trajectory curvature feature and the speed change rate feature , the direction compensation amount and the height compensation amount are calculated, so as to obtain the trajectory correction parameter of the unmanned aerial vehicle, the trajectory correction parameter including the direction correction amount , the speed correction amount and the height correction amount , and the calculation process is as follows: ； ； ； ； ； wherein, is a curvature-velocity coupling coefficient, is a safety envelope height, is a height tolerance coefficient, is a predicted height, is a predicted heading angle, is a predicted velocity, is a velocity correction gain; The process of converting the trajectory correction parameters into unmanned aerial vehicle correction instructions includes: When three times in a row , the deviation is determined to be persistent, triggering a correction instruction, when not more than three times in a row , only a log is recorded, without triggering a control instruction, the correction instruction comprising a direction correction instruction , a speed correction instruction and a height correction instruction , the calculation of which is as follows: ； ； ； ； wherein, is a proportional coefficient, is an integral coefficient, is the deviation of the heading angle for the last three times, is the deviation of the speed, is the current actual speed, is a speed tracking coefficient, is an altitude tracking coefficient; The trajectory monitoring module constructs a visual interface including a predicted trajectory line, an actual trajectory line and a safety envelope surface according to the attitude adjustment feedback. 2.The multi-sensor fusion based UAV trajectory tracking system of claim 1, wherein: In the unmanned aerial vehicle data acquisition module, the acquisition and preprocessing process of the unmanned aerial vehicle data and the GPS data includes: The unmanned aerial vehicle data includes three-axis acceleration and angular velocity data of the unmanned aerial vehicle, and the GPS data includes GPS latitude and longitude coordinates and velocity scalar data; The inertial measurement unit is fixed on the gravity center plane of the unmanned aerial vehicle, and is arranged in a three-axis orthogonal layout and aligned with the body coordinate system. The inertial measurement unit measures three-axis acceleration data in the body coordinate system through a three-axis accelerometer, and measures angular velocity data around the body coordinate axis through a three-axis gyroscope. The collected three-axis acceleration and angular velocity data of the unmanned aerial vehicle are cached in a ring queue with a queue length of 50 groups according to the timestamp. The global positioning system receiver is arranged on the top of the unmanned aerial vehicle, and the antenna faces the unobstructed area of the sky. The global positioning system receiver receives satellite signals to calculate latitude and longitude coordinates and velocity scalar data. The collected latitude and longitude coordinates and velocity scalar data are parsed according to the protocol and stored in a first-in-first-out buffer with a buffer capacity of 20 groups. The inertial measurement unit is directly connected to the main control module of the unmanned aerial vehicle trajectory tracking system through a four-wire serial bus, the sampling frequency is set to 200 Hz, a shock absorbing base is added, the global positioning system receiver uses an asynchronous serial communication protocol, the sampling frequency is 10 Hz, and the clock signal is synchronized with the inertial measurement unit, and a built-in backup power supply is provided. High-frequency noise of the three-axis acceleration and angular velocity data of the unmanned aerial vehicle is suppressed by Kalman filtering technology. The state transition matrix and the observation noise covariance matrix are designed to iteratively predict and update the true values of acceleration and angular velocity. The filtered data are aligned by timestamp and cached in the ring queue, and the three-axis acceleration and angular velocity data after noise suppression are output. The sliding window mean value filtering is adopted, the latitude and longitude coordinates and the speed scalar in the window are respectively calculated, the data points with the direction difference of the adjacent points of the speed vector exceeding 30 degrees are removed and replaced by the median in the window, meanwhile, the buffer interpolation algorithm is enabled in the signal interruption period, the missing values are linearly extrapolated by using the historical data, and the smoothed latitude and longitude coordinates and speed scalar data are output. 3.The multi-sensor fusion based UAV trajectory tracking system of claim 2, wherein: The feature extraction process of the preprocessed UAV data in the UAV feature extraction module comprises: The time domain variance features of the acceleration and angular velocity data of each axis in the preprocessed window are obtained, and three-axis acceleration variance and three-axis angular velocity variance are obtained; The three-axis acceleration and angular velocity data of the same window are respectively subjected to fast Fourier transform, the frequency domain amplitude spectrum is obtained, the three frequency points with the maximum amplitude in the frequency domain are extracted, the proportion of the energy of the three frequency points to the total energy of all frequency points in the Nyquist frequency is analyzed, and the frequency domain principal component energy proportion features of the acceleration and angular velocity are output.

4. The multi-sensor fusion based UAV trajectory tracking system of claim 3, wherein: The feature extraction process of the preprocessed GPS data in the UAV feature extraction module comprises: Based on the latitude and longitude coordinates of the three consecutive points in the preprocessed sliding window, the latitude and longitude coordinates are converted into plane coordinates, the curvature radius reciprocal of the circular arc formed by the three points is analyzed, and the trajectory curvature feature of the current window is output; Based on the preprocessed speed scalar data, the heading angle of the sampling point is analyzed according to the difference between the adjacent latitude and longitude coordinates, the length change of the speed vector of the adjacent sampling points is obtained, and the speed change rate feature is obtained; The time domain variance, frequency domain principal component energy proportion, trajectory curvature and speed change rate are aligned by time stamp interpolation, and are integrated into a trajectory tracking feature sequence list.

5. The multi-sensor fusion based UAV trajectory tracking system of claim 4, wherein: In the trajectory prediction deviation scoring module, the trajectory prediction model is constructed in combination with the trajectory tracking feature sequence list, and the process of outputting the three-dimensional trajectory prediction coordinates comprises: A stacked double memory unit layer based on a long short-term memory network architecture is adopted, the first layer and the second layer each include 32 memory units, 10 groups of data are taken as a time window, the time domain variance features and the speed change rate features are combined into a 7-dimensional prediction sequence, each feature is respectively normalized by maximum-minimum, and the trajectory prediction model is constructed; The prediction sequence is input into the long short-term memory network model in sequence according to the time step, the information flow is controlled through the internal forgetting gate and the input gate, the long-term dependence relationship between the time sequence features is stored in the memory unit, and the hidden state output at the last time step is mapped into three-dimensional trajectory prediction coordinates through a full connection layer, the three-dimensional trajectory prediction coordinates include the longitude coordinate, the latitude coordinate and the height coordinate of the UAV; The mean square error is used to measure the difference between the three-dimensional trajectory prediction coordinates and the real trajectory coordinates, the partial derivatives of the loss function with respect to the weights and biases of each layer are calculated by the gradient descent method, the maximum number of iterations is set to 5000 times, and the training is terminated when the maximum number of iterations is reached. 6.The multi-sensor fusion based UAV trajectory tracking system of claim 1, wherein: In the trajectory dynamic correction module, the process of generating the attitude adjustment feedback quantity comprises: The proportional-integral-derivative controller is based on the difference between the correction amount and the corresponding actual measured value 、 and The heading angle error, the speed error and the height error are obtained in real time, the control weights are dynamically adjusted in combination with the proportional term, the integral term and the derivative term, the derivative term suppresses high-frequency oscillation, the integral term eliminates steady-state deviation, and the attitude adjustment feedback amount is output, the attitude adjustment feedback amount including the rudder adjustment amount, the throttle adjustment amount and the elevator adjustment amount.

7. The multi-sensor fusion based UAV trajectory tracking system of claim 6, wherein: In the trajectory monitoring module, the process of constructing the visual interface comprising the predicted trajectory line, the actual trajectory line and the safety envelope surface according to the attitude adjustment feedback quantity comprises: The trajectory prediction coordinates output by the integrated trajectory prediction model, the smoothed trajectory of the global positioning system, and the safety envelope surface parameters are converted into three-dimensional space coordinates by plane projection, and a red predicted trajectory line, a blue actual trajectory line, and a semi-transparent green cylindrical safety envelope surface are constructed. The interface is refreshed at a period of 0.1 seconds, and real-time attitude adjustment feedback values are displayed synchronously. Zooming, rotating, and trajectory feature parameter browsing functions are supported. When the actual trajectory point continuously exceeds the safety envelope surface, a secondary warning mechanism is triggered, a primary warning mark deviates from the point and a pop-up window is prompted, and a secondary warning forces the switching of the control mode. The operator can manually adjust the envelope surface tolerance parameters and click the trajectory line to view the time domain variance and the frequency domain principal component energy proportion.

Images