A method for predicting the trajectory of low-altitude unmanned aerial vehicles based on multi-source data

Abstract

This invention relates to the field of trajectory tracking and discloses a low-altitude unmanned aerial vehicle (UAV) trajectory prediction method based on multi-source data, which significantly improves the robustness and accuracy of trajectory tracking. The method includes processing source phased array radar observation data and visual sensor image sequences separately to generate radar and visual state probability distributions. These distributions are then aligned in a common state space to obtain a subspace probability distribution, which is then mapped to the same statistical manifold to determine the corresponding manifold points. Based on the real-time measurement quality of the sensors, confidence weights are dynamically determined. After weighting the manifold points, a fused manifold point is obtained, which is mapped back to generate a fused state probability distribution. The optimal motion state estimate is extracted from the fused distribution, and multi-step iterative prediction is performed based on the UAV dynamics model to output a predicted trajectory sequence for multiple discrete time points in the future. This invention effectively fuses multi-source data, improving the accuracy and robustness of trajectory tracking and meeting the flight safety and management requirements of low-altitude UAVs.

Description

Technical Field

[0001] This invention relates to the field of trajectory tracking, and more particularly to a method for predicting the trajectory of a low-altitude unmanned aerial vehicle (UAV) based on multi-source data. Background Technology

[0002] In low-altitude airspace, drones are increasingly widely used, covering numerous fields such as logistics delivery, environmental monitoring, agricultural plant protection, and emergency rescue. However, with the rapid increase in the number of low-altitude drones, their flight safety and management issues are becoming increasingly prominent. Accurate and real-time tracking of the trajectories of low-altitude drones is crucial for ensuring flight safety, avoiding collisions, maintaining air traffic order, and achieving effective supervision and dispatch.

[0003] Currently, single-sensor systems have many limitations in tracking low-altitude unmanned aerial vehicles (UAVs). Phased array radars offer advantages such as long detection range and immunity to lighting and weather conditions, enabling them to acquire three-dimensional information like distance and speed. However, their ability to capture subtle motion characteristics of UAVs is limited, and they are susceptible to interference in complex electromagnetic environments. Visual sensors can provide rich image information, intuitively identifying the UAV's appearance and tracking its movement. However, they are easily affected by changes in lighting, occlusion, and inclement weather, leading to degraded image quality and consequently impacting tracking accuracy.

[0004] Sensor data inherently contains a degree of uncertainty, and current technologies are insufficient in handling this uncertainty when fusing multi-source data. They fail to adequately consider the uncertainty parameters of different sensors, such as the covariance matrix, leading to inaccurate state estimations after fusion and an inability to provide a reliable basis for trajectory tracking. Some existing multi-source data fusion methods have high computational complexity, requiring a lot of computing resources and time, making it difficult to meet the real-time requirements of low-altitude UAV trajectory tracking. Some fusion methods based on complex mathematical models and algorithms have slow computation speed when processing large amounts of sensor data, and cannot output trajectory tracking results in a timely manner. Current airspace situation analysis often requires operators to simultaneously observe radar traces and video footage, relying on experience to judge the aircraft's movement intentions. This not only demands extremely high professional skills from personnel, but also makes it prone to prediction errors due to human fatigue or misjudgment when facing multi-target, highly dynamic drone swarm activities, failing to meet the needs of automated and intelligent airspace supervision.

[0005] Therefore, we propose a low-altitude UAV trajectory prediction method based on multi-source data to solve the above problems. Summary of the Invention

[0006] This invention provides a low-altitude UAV trajectory prediction method based on multi-source data, which significantly improves the robustness and accuracy of trajectory tracking.

[0007] The first aspect of this invention provides a method for predicting the trajectory of a low-altitude unmanned aerial vehicle (UAV) based on multi-source data. The method includes: processing observation data from a source phased array radar to generate a radar state probability distribution; processing image sequences from a visual sensor to generate a visual state probability distribution through moving target detection and feature tracking; mapping the radar state probability distribution and the visual state probability distribution to the same statistical manifold to obtain two corresponding manifold points; weighting the two manifold points based on sensor confidence and minimizing the geometric distance on the manifold to obtain a fused state probability distribution; and extracting the optimal motion state estimate of the UAV from the fused state probability distribution for trajectory prediction and output.

[0008] Optionally, in a first implementation of the first aspect of the present invention, the method includes: acquiring observation data including a three-dimensional point cloud and a micro-Doppler matrix from a source phased array radar; performing image morphological filtering and connected component analysis on the micro-Doppler matrix to extract stable micro-motion features and generate a radar feature vector; combining the position and velocity information of the three-dimensional point cloud with the radar feature vector to construct a radar state vector; and generating a radar state probability distribution based on the radar state vector and a preset radar measurement noise model.

[0009] Optionally, in a second implementation of the first aspect of the present invention, the method includes: acquiring an image sequence collected by a visual sensor; performing motion target detection on the image sequence based on optical flow calculation to obtain a target region including a UAV; extracting multiple image feature points within the target region and tracking the motion of the image feature points in the sequence to generate a two-dimensional feature point trajectory set; back-projecting the two-dimensional feature point trajectory set according to a camera imaging model to obtain a corresponding three-dimensional spatial point motion trajectory set; performing statistical analysis on the three-dimensional spatial point motion trajectory set to obtain a visual state vector characterizing the motion state of the UAV; and generating a visual state probability distribution based on the visual state vector and a preset visual measurement error model.

[0010] Optionally, in a third implementation of the first aspect of the present invention, the method includes: extracting a radar state vector and a radar uncertainty parameter from the radar state probability distribution; extracting a visual state vector and a visual uncertainty parameter from the visual state probability distribution; converting the radar state vector and radar uncertainty parameter, as well as the visual state vector and visual uncertainty parameter, into coordinate expressions on a statistical manifold, respectively; and determining corresponding radar manifold points and visual manifold points on the statistical manifold based on the coordinate expressions.

[0011] Optionally, in a fourth implementation of the first aspect of the present invention, the method includes: calculating a corresponding radar Fisher information matrix based on the covariance matrix in the radar uncertainty parameters; calculating a corresponding visual Fisher information matrix based on the covariance matrix in the visual uncertainty parameters; combining the radar state vector with the radar Fisher information matrix to form a complete geometric expression of the radar probability distribution, which serves as the coordinates of the radar manifold point; and combining the visual state vector with the visual Fisher information matrix to form a complete geometric expression of the visual probability distribution, which serves as the coordinates of the visual manifold point.

[0012] Optionally, in the fifth implementation of the first aspect of the present invention, the method includes: dynamically determining the radar confidence weight and the visual confidence weight based on the real-time measurement quality of the source phased array radar and the visual sensor; weighting the geometric representations of the radar manifold points and the visual manifold points on the statistical manifold based on the radar confidence weight and the visual confidence weight to obtain a weighted manifold point representation; finding a fusion point on the statistical manifold through optimization calculation such that the combined geometric distance from the fusion point to the weighted radar manifold point and the visual manifold point is minimized to obtain a fused manifold point; and mapping the fused manifold point back to the probability distribution parameter space to generate a fused state probability distribution.

[0013] Optionally, in the sixth implementation of the first aspect of the present invention, the global weights are calculated using the range normalization exponent formula: ; in, Environmental penalty index; For the real-time signal-to-noise ratio of the radar; This represents the minimum radar signal-to-noise ratio. This represents the maximum value of the radar signal-to-noise ratio; Variance of real-time visual sharpness; This represents the minimum value of the variance in visual acuity; This represents the maximum value of the variance in visual acuity.

[0014] Optionally, in the seventh implementation of the first aspect of the present invention, the method includes: parsing the optimal state vector and the corresponding fusion covariance matrix at the current moment from the fusion state probability distribution; constructing a state transition matrix based on a preset UAV dynamics model; using the optimal state vector as the initial state, performing multi-step iterative prediction using the state transition matrix to obtain a sequence of predicted state vectors for multiple discrete time points in the future, while simultaneously propagating the fusion covariance matrix through the state transition matrix and the process noise matrix to obtain a corresponding sequence of predicted covariance; combining the sequence of predicted state vectors with the sequence of predicted covariance to generate and output a predicted trajectory sequence.

[0015] Optionally, in an eighth implementation of the first aspect of the present invention, the radar state probability distribution and the visual state probability distribution are aligned in a common state space to obtain a radar subspace probability distribution and a visual subspace probability distribution, respectively; the common state space is a space composed of position and velocity components.

[0016] Optionally, in a ninth implementation of the first aspect of the present invention, a first multidimensional state vector and a first covariance matrix are extracted from the radar state probability distribution; a first projection transformation matrix is ​​constructed according to the dimension definition of the common state space; the first multidimensional state vector is dimensionality-reduced by projecting the first projection transformation matrix to obtain a radar subspace state vector, and the first covariance matrix is ​​simultaneously transformed to obtain a radar subspace covariance matrix; the radar subspace state vector and the radar subspace covariance matrix are combined to form the radar subspace probability distribution; and the same projection transformation operation is performed on the visual state probability distribution based on the common state space of the same dimension to generate a visual subspace probability distribution.

[0017] The mechanism of this invention is as follows: at a more fundamental probability distribution level, the observation data of source phased array radar and vision are geometrically represented and fused. Geometric quantities such as Fisher information matrix are used to accurately characterize sensor uncertainty, and environmental adaptive weights are combined to optimize on the manifold, thereby effectively solving the problem of difficult deep fusion of heterogeneous data and uncertainty transmission in complex environments. Beneficial effects: Based on the real-time measurement quality of the source phased array radar and visual sensors, the radar confidence weight and visual confidence weight are dynamically determined. The global weight is calculated using the range normalization exponent formula. It comprehensively considers factors such as the real-time signal-to-noise ratio of the radar and the variance of the real-time visual sharpness. It can reasonably allocate the weight of each sensor data according to the sensor performance under different environmental conditions, give full play to the advantages of each sensor, and improve the reliability of the fusion results. Uncertainty parameters are extracted from radar state probability distribution and visual state probability distribution, and the corresponding Fisher information matrix is ​​calculated. The state vector and Fisher information matrix are combined to form a complete geometric expression of the probability distribution, which is used as coordinates on the statistical manifold. In the fusion process, the uncertainty of sensor data is fully considered. By minimizing the geometric distance on the manifold, a more accurate fused state probability distribution is obtained, which provides a reliable state estimate for trajectory tracking. Data fusion is performed on the statistical manifold, and the fusion point is found through optimized calculation to minimize the comprehensive geometric distance. The calculation process is relatively simple and efficient. In the trajectory prediction stage, a state transition matrix is ​​constructed based on the preset UAV dynamics model, and multi-step iterative prediction is performed. It can output accurate predicted trajectory sequences while meeting real-time requirements. Existing shallow fusion methods ignore the dynamic changes in sensor performance with the environment, leading to abrupt changes in prediction results and forcing staff to repeatedly verify them. This solution designs a weighted fusion mechanism based on sensor confidence, using the range normalization exponent formula to calculate the dynamic weights of radar signal-to-noise ratio and visual sharpness variance in real time. It can automatically adjust the contribution of radar and vision in fusion according to environmental changes. The reliable results directly reduce the frequency of manual verification and improve the overall work efficiency from situational awareness to trajectory prediction to administrative decision-making. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of an embodiment of the low-altitude UAV trajectory prediction method based on multi-source data in this invention.

[0019] Figure 2 This is a schematic diagram of another embodiment of the low-altitude UAV trajectory prediction method based on multi-source data in this invention.

[0020] Figure 3 A schematic diagram illustrating the generation of a visual state vector for the position and velocity of a UAV by combining an image sequence captured by a visual sensor with a depth reference provided by radar.

[0021] Figure 4 This is a schematic diagram of an embodiment of a low-altitude UAV trajectory tracking device based on multi-source data in this invention. Detailed Implementation

[0022] This invention provides a low-altitude unmanned aerial vehicle (UAV) trajectory prediction method based on multi-source data, which significantly improves the robustness and accuracy of trajectory tracking. The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0023] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the low-altitude UAV trajectory prediction method based on multi-source data in this invention includes: 101. Process the observation data of the source phased array radar and generate the radar state probability distribution; It is understood that the executing entity of this invention can be a low-altitude UAV trajectory tracking device based on multi-source data, or it can be a terminal or a server; the specific implementation is not limited here. This embodiment of the invention will be described using a server as an example.

[0024] It should be noted that the ground-based phased array radar detected a low-flying micro quadcopter UAV at time T0. The polar coordinate observation data output by the radar's underlying signal processing unit were: target slant range 1500 meters, azimuth 45.0 degrees, and elevation 10.0 degrees. Simultaneously, Doppler frequency shift calculations determined the target's radial approach velocity to be 15.0 meters per second.

[0025] The prior error parameters of the radar under the current operating conditions were retrieved: the standard deviation of the ranging error is 5 meters, the standard deviation of the angle measurement error (azimuth and elevation) is 0.5 degrees, and the standard deviation of the velocity measurement error is 1.0 meter per second. These physical errors form the basis for the subsequent construction of the probability distribution.

[0026] To unify the reference with other sensors, the system must transform the polar coordinate system into a three-dimensional Cartesian coordinate system with a north-south orientation. Through rigorous nonlinear spatial geometric calculations, the mean values ​​of the target's current state were established: Mean position: approximately 1044.5 meters east, approximately 1044.5 meters north, and approximately 260.4 meters vertically. Mean velocity: The system combines radial velocity with spatial angular decomposition and superimposes the azimuth change trends from two consecutive radar frames to derive the target's three-dimensional motion rate in the Cartesian coordinate system as follows: 11.0 meters per second westward (negative eastward), 9.0 meters per second southward (negative northward), and slowly ascending, with a vertical upward velocity of 1.0 meter per second.

[0027] By utilizing a nonlinear mapping (differentiation of the Jacobian matrix), the scalar measurement error in polar coordinates is transferred to a three-dimensional Cartesian coordinate system, resulting in the calculation of a six-dimensional (three position dimensions and three velocity dimensions) covariance matrix. Physically, this position covariance matrix manifests as a flattened error ellipsoid: in the radial (depth) direction pointing towards the radar, the ellipsoid is extremely narrow (with minimal error) due to high ranging accuracy; however, in the transverse section perpendicular to the line of sight, the ellipsoid is very wide because a 0.5-degree angular measurement error is amplified to tens of meters at a distance of 1500 meters.

[0028] Using the calculated three-dimensional state vector (position and velocity) as the expected mean of a Gaussian distribution and the six-dimensional covariance matrix as the variance feature, a complete radar state probability distribution model is constructed. This distribution accurately describes the target's physical state and its divergence characteristics as observed by the radar in mathematical terms.

[0029] 102. Process image sequences from visual sensors and generate visual state probability distributions through moving target detection and feature tracking; It should be noted that at time T0, the telephoto vision sensor outputs an ultra-high-definition image sequence. The target detection algorithm locks onto the drone in the image, calculates its pixel center coordinates, and converts them into physical line-of-sight angles using a perspective projection model: azimuth 45.1 degrees and pitch 9.8 degrees (with a slight but reasonable systematic deviation from radar detection).

[0030] The target's pixel displacement over the past 5 frames is continuously tracked using an optical flow algorithm. The pixel movement rate is combined with the camera's focal length to convert it into the target's physical angular velocity: the azimuth angle changes at -0.4 degrees per second, and the pitch angle changes at 0.05 degrees per second.

[0031] Since monocular vision cannot directly measure distance, the system must respect this physical fact and cannot forcibly borrow radar data. When constructing the visual state mean, the system sets a very broad prior distance mean (preset to 1000 meters) in the line-of-sight direction. Combining this with an azimuth angle of 45.1 degrees and a pitch angle of 9.8 degrees, it calculates the mean three-dimensional position of the target at this moment: approximately 696.5 meters due east, approximately 694.7 meters due north, and approximately 170.2 meters in altitude. Based on the calculated angular velocity, the system infers the mean three-dimensional velocity components of the target's vision at this moment as follows: 7.5 meters per second to the west, 6.5 meters per second to the south, and an altitude gain of 0.8 meters per second.

[0032] The pixel error is set based on the camera resolution (corresponding to an extremely small angular error of approximately 0.05 degrees), but the standard deviation of the measurement error in the depth direction is set to an extremely large value (10,000 meters). The generated visual covariance matrix appears in physical space as an extremely slender ray-shaped cone or an infinitely extended, extremely elongated ellipsoid. In the lateral section, its error is less than 1 meter; but in the depth direction, its error is almost infinite.

[0033] The mean values ​​of the aforementioned position and velocity, along with this special, extremely elongated covariance matrix, are collectively encapsulated into a visual state probability distribution. This distribution faithfully reflects the characteristics of a visual sensor: it is very certain that the target is on this extremely thin ray and knows how fast it is moving laterally, but it has absolutely no idea of ​​its exact distance from this ray.

[0034] 103. Map the radar state probability distribution and the visual state probability distribution to the same statistical manifold to obtain two corresponding manifold points; It should be noted that at this time, the system memory simultaneously maintains two complex Gaussian distributions in a unified three-dimensional Cartesian coordinate system: Radar probability distribution parameters, including the mean position (East 1044.5, North 1044.5, Altitude 260.4), the mean velocity (West 11.0, South 9.0, Ascent 1.0), and a flattened six-dimensional covariance matrix. Visual probability distribution parameters, including the mean position (East 696.5, North 694.7, Altitude 170.2), the mean velocity (West 7.5, South 6.5, Ascent 0.8), and a ray-shaped six-dimensional covariance matrix with a large depth variance.

[0035] Each of these mean vectors plus covariance matrices can be considered as an indivisible, holistic set of feature parameters.

[0036] The underlying algorithm mathematically constructs a high-dimensional Riemannian manifold space. In this curved statistical space, each point no longer represents a specific location in the physical world (such as a certain number of meters), but rather a Gaussian probability distribution model with a specific mean and variance. The curvature of this manifold is directly determined by the amount of information in the distribution (the Fisher information matrix); the smaller the dimension with smaller variance, the stronger the surface stretching caused by this manifold space.

[0037] By utilizing the exponential mapping relationship between Lie groups and Lie algebras, the system performs coordinate substitution.

[0038] Substituting the radar probability distribution parameter package into the mapping equation and projecting it onto this curved surface yields a fixed geometric point on the surface, denoted as the radar manifold point. Using the exact same mapping equation, the visual probability distribution parameter package is also projected onto the same surface, yielding another geometric point with a certain spatial distance from the radar point, denoted as the visual manifold point. Through this step, two error probability clouds that were originally physically different and could not be directly added are elegantly transformed into two standard geometric points on a curved mathematical surface.

[0039] 104. Weight the two manifold points based on sensor confidence and calculate a fusion state probability distribution by minimizing the geometric distance on the manifold; It should be noted that in the manifold space, the weights are naturally determined by the inverse of the covariance matrix—that is, the Fisher information matrix.

[0040] Radar exhibits extremely low variance in the depth dimension, resulting in a massive amount of information; conversely, vision exhibits extremely low variance in the lateral angle dimension, resulting in a massive amount of lateral information. The system automatically performs matrix operations, assigning extremely high weights to radar in depth calculations and extremely high weights to vision in angle and lateral displacement calculations. These weights are interwoven, dynamic, and distinct across six dimensions, far exceeding the expressive power of a single scalar.

[0041] On a curved Riemannian manifold, the shortest path connecting radar manifold points and visual manifold points is a nonlinear curve called a geodesic. The system searches along this geodesic for an optimal point that minimizes the global information divergence (Kullback-Leibler divergence, or geometric distance).

[0042] Iterative calculations are performed using a gradient descent algorithm on the manifold. Due to the curvature of the manifold, the found fused manifold point does not fall on the center of the physical straight line connecting the two original points. It is forcibly pulled toward the radar data domain on the depth axis, while being tightly attached to the trajectory of the visual ray on the lateral angular axis.

[0043] After locating the fusion manifold point, the system performs a logarithmic mapping (inverse mapping) to unpack it back into a physical probability distribution in three-dimensional Cartesian space.

[0044] Due to the nonlinear compression effect of manifold geometry, the average three-dimensional position calculated by fusion is: 1043.2 meters due east, 1046.8 meters due north, and an altitude of 258.9 meters. This coordinate is not a simple linear weighted average of the radar and visual averages (a simple weighted average altitude would not be 258.9 meters), but rather profoundly reflects the nonlinear optimal geometric intersection point that takes the radar distance and the visual angle.

[0045] Similarly, the velocity mean was precisely fused to: 11.8 m / s westward, 8.5 m / s southward, and 1.2 m / s ascent. More importantly, the volume of the fused new covariance matrix drastically decreased, becoming an extremely compact sphere. This means that the system completely eliminated radar angle errors and visual depth errors, achieving state estimation with extremely high confidence.

[0046] 105. Extract the optimal motion state estimate of the UAV from the fused state probability distribution, and predict and output the trajectory accordingly.

[0047] It should be noted that from the high-dimensional fused Gaussian distribution obtained in step 104, the system directly extracts the mean vector with the probability density at its peak, and uses it as the optimal motion state of the UAV in the physical world at the current time T0. After removing the extremely small covariance error band, the system establishes the specific baseline data at time T0: Current optimal position: East coordinate 1043.2 meters, North coordinate 1046.8 meters, Vertical altitude 258.9 meters. Current optimal speed: East speed -11.8 m / s (i.e., westward), North speed -8.5 m / s (i.e., southward), Vertical speed +1.2 m / s (i.e., climbing).

[0048] This state not only integrates Doppler velocimetry and visual optical flow angular velocity, but also undergoes depth alignment on the manifold, representing the highest accuracy true value within the current capability range of the system.

[0049] Given that the target is a typical low-altitude micro-rotor UAV, its aerodynamic characteristics dictate that it cannot perform drastic maneuvering changes within a short period (on the order of seconds). Therefore, the system activates a three-dimensional constant velocity motion model (CV model) as the prediction engine.

[0050] The system uses the optimal position at time T0 as the initial anchor point, and uses the extracted optimal three-dimensional velocity components as the first derivative, performing recursive integration on the time axis at a fixed time step (set to 1 second). The calculation logic is rigorous and conforms to the laws of physics: the position in the next second is equal to the current position plus the velocity vectors in their respective dimensions.

[0051] Through continuous model simulation, the system generates predicted 3D waypoints for three consecutive key nodes within the next 3 seconds. To ensure seamless data integration with downstream air defense fire control computing platforms or visualization terminals, the system packages and encapsulates the current status and predicted trajectory to generate a standard data list.

[0052] The detailed analysis of the optimal state and trajectory prediction output data is shown in Table 1 below: Table 1 Through the above derivation, the system completed a complete data iteration and successfully output the UAV's 3D trajectory, which combines high-precision positioning and reliable forward prediction.

[0053] Please see Figure 2 and Figure 3 Another embodiment of the low-altitude UAV trajectory prediction method based on multi-source data in this invention includes: 201. Process the observation data of the source phased array radar and generate the radar state probability distribution; Specifically, observation data including three-dimensional point clouds and micro-Doppler matrices are acquired from the source phased array radar; image morphological filtering and connected component analysis are performed on the micro-Doppler matrix to extract stable micro-motion features and generate radar feature vectors; the position and velocity information of the three-dimensional point cloud and the radar feature vectors are combined to construct radar state vectors; and radar state probability distributions are generated based on the radar state vectors and a preset radar measurement noise model.

[0054] It should be noted that a high-frequency phased array radar detected a low-altitude quadcopter UAV at time T0. The radar front-end output two types of core data: 3D point cloud data: the target's current spatial position is anchored at 1200 meters due east, 800 meters due north, and an altitude of 150 meters; the macroscopic flight speed is 10 m / s eastward, 5 m / s southward (-5 m / s northward), and 0 m / s vertically. Micro-Doppler matrix: the system generates a time-frequency two-dimensional image matrix including the characteristics of the UAV's rotor cutting through the air at high speed.

[0055] After performing image morphological opening filtering on the micro-Doppler matrix to eliminate environmental noise, connected component analysis was performed to extract the rotor micro-motion envelope. By analyzing the period and amplitude, the system successfully extracted two microscopic physical parameters: rotor speed frequency of 60 Hz and blade tip linear velocity of 45 m / s. These two values ​​together constitute the radar feature vector at this point.

[0056] The six macroscopic kinematic parameters and two micro-motion features are concatenated to construct a complete 8-dimensional radar state vector (i.e., the system's predicted mean). Then, a preset radar measurement noise model is loaded: assuming a position error standard deviation of 2 meters, a velocity error of 0.5 m / s, a frequency resolution error of 1.5 Hz, and a propeller tip velocity error of 1 m / s under the current operating conditions. The system generates an 8th-order covariance matrix based on these error boundaries. Finally, the 8-dimensional state vector and this covariance matrix are encapsulated to output a high-confidence radar state probability distribution.

[0057] 202. Process image sequences from the vision sensor and generate a visual state probability distribution through moving target detection and feature tracking; Specifically, the process involves acquiring image sequences from a visual sensor; performing motion target detection based on optical flow calculation on the image sequences to obtain a target region including the UAV; extracting multiple image feature points within the target region and tracking the movement of these feature points in the sequence to generate a set of two-dimensional feature point trajectories; back-projecting the set of two-dimensional feature point trajectories using a camera imaging model to obtain a corresponding set of three-dimensional spatial point motion trajectories; performing statistical analysis on the set of three-dimensional spatial point motion trajectories to calculate a visual state vector representing the UAV's motion state; and generating a visual state probability distribution based on the visual state vector and a pre-defined visual measurement error model.

[0058] It should be noted that the optoelectronic pod outputs a high-definition image sequence at 60 frames per second. The system utilizes an optical flow algorithm to compare pixel abrupt changes between adjacent frames, successfully locking onto an extremely minute motion contour in the background at a distance of 1200 meters. Consistent with the physics of long-distance imaging, the pixel group size of this target area is only approximately 12 by 6 pixels. The system extracts multiple corner features within this tiny region and continuously tracks its sub-pixel-level displacement over 10 consecutive frames, generating a set of two-dimensional feature point trajectories.

[0059] Monocular vision inherently lacks absolute depth. To avoid a common-sense depth error of hundreds of meters caused by pure visual back-projection, the system introduces an approximate depth reference plane (approximately 1200 meters) provided by radar as a strong geometric constraint. Based on the camera's internal perspective model and this reference plane, the system reverse-tracks the two-dimensional pixel trajectories into ray tracings, reconstructing them into three-dimensional spatial point motion trajectories.

[0060] After filtering and smoothing the 3D trajectory, the system calculates the visual state vector: position is 1202.5 meters east, 798.5 meters north, and altitude is 151.0 meters; speed is 9.8 m / s eastward, 5.2 m / s southward, and 0.1 m / s ascent. Simultaneously, 12x6 pixels are extracted as visual features and concatenated into an 8-dimensional vector.

[0061] Loading the visual error model: With the assistance of the radar depth plane, the depth error is compressed to a standard deviation of 8 meters, while the lateral position error is extremely small (standard deviation of 1.5 meters). Based on this, the system generates an 8th-order covariance matrix and combines it with the state vector to output a visual state probability distribution that includes strong directional error characteristics.

[0062] 203. Map the radar state probability distribution and the visual state probability distribution to the same statistical manifold to obtain two corresponding manifold points; Specifically, radar state vectors and radar uncertainty parameters are extracted from radar state probability distribution; visual state vectors and visual uncertainty parameters are extracted from visual state probability distribution; radar state vectors and radar uncertainty parameters, as well as visual state vectors and visual uncertainty parameters, are converted into coordinate expressions on statistical manifolds; based on the coordinate expressions, corresponding radar manifold points and visual manifold points are determined on the statistical manifolds.

[0063] Furthermore, the radar state vector and radar uncertainty parameters, as well as the visual state vector and visual uncertainty parameters, are converted into coordinate representations on a statistical manifold, including: calculating the corresponding radar Fisher information matrix based on the covariance matrix in the radar uncertainty parameters; calculating the corresponding visual Fisher information matrix based on the covariance matrix in the visual uncertainty parameters; combining the radar state vector and the radar Fisher information matrix to form a complete geometric representation of the radar probability distribution, which serves as the coordinates of the radar manifold points; and combining the visual state vector and the visual Fisher information matrix to form a complete geometric representation of the visual probability distribution, which serves as the coordinates of the visual manifold points.

[0064] It should be noted that the 6-dimensional radar subspace probability distribution and visual subspace probability distribution preprocessed and output by step 206 can be received.

[0065] The 6th-order covariance matrices of each are extracted, and the corresponding Fisher information matrices are calculated by matrix inversion. The Fisher matrix perfectly quantifies the certainty of measurement mathematically: the Fisher matrix of radar has a maximum value in the depth dimension (eastward direction) (the square of the reciprocal of the error of 2 meters), while the Fisher matrix of vision has a maximum value in the lateral dimension (northward direction) (the square of the reciprocal of the error of 1.5 meters).

[0066] By deeply binding the radar's 6-dimensional state vector with its corresponding Fisher information matrix, a complete geometric representation of the radar's probability distribution in the manifold space is formed. Similarly, the visual state vector and the visual Fisher information matrix are also bound together.

[0067] Substituting these two sets of geometric coordinates into the underlying Riemannian statistical surface, the two originally divergent error probability clouds are abstracted and fixed without loss and with rigor into two clearly defined endpoints on the curved surface: namely, the radar manifold point and the visual manifold point, paving the way for optimal fusion.

[0068] 204. Weight the two manifold points based on sensor confidence and calculate a fusion state probability distribution by minimizing the geometric distance on the manifold; Specifically, based on the real-time measurement quality of the source phased array radar and the visual sensor, the radar confidence weight and the visual confidence weight are dynamically determined. Based on the radar confidence weight and the visual confidence weight, the geometric representations of the radar manifold points and the visual manifold points on the statistical manifold are weighted to obtain the weighted manifold point representation. On the statistical manifold, a fusion point is found through optimization calculations such that the combined geometric distance from the fusion point to the weighted radar manifold point and the visual manifold point is minimized, thus obtaining the fused manifold point. The fused manifold point is mapped back to the probability distribution parameter space to generate the fused state probability distribution.

[0069] Furthermore, based on the real-time measurement quality of the source phased array radar and the visual sensor, the radar confidence weight and the visual confidence weight are dynamically determined, including: obtaining the real-time signal-to-noise ratio index of the source phased array radar and the real-time image sharpness index of the visual sensor respectively; converting the real-time signal-to-noise ratio index into a radar quality factor and the real-time image sharpness index into a visual quality factor according to a predefined mapping relationship; and generating the radar confidence weight and the visual confidence weight respectively through normalization calculation based on the radar quality factor and the visual quality factor.

[0070] It should be noted that obtaining the real-time signal-to-noise ratio of the radar... dB (range 10 to 30 dB), and real-time visual sharpness variance (Interval 40 to 100). To eliminate the dimensional barrier between logarithmic units and conventional numerical values, the range normalization exponent formula is used to calculate the global weights: Set environmental penalty index The calculated molecule is... The denominator of the visual factor is The pure scalar global weights are derived: Radar Visual .

[0071] Rigorous manifold optimization and self-consistent inverse mapping solution, on the manifold, the system utilizes and As a global multiplier, combined with the Fisher information matrix (local curvature tensor), the fusion point that minimizes the integrated Fréchet distance is found along the geodesic and then mapped back to physical space.

[0072] Taking the due north direction (lateral) as an example, the radar measurement is 800 meters (variance 4), while the visual measurement is 798.5 meters (variance 2.25). The effective information contribution ratio of radar in this dimension is... Visual The north coordinates are integrated, strictly adhering to the visual input based on the amount of information required. rice.

[0073] In the due east direction (depth), the visual variance is extremely large (64), with only about 0.01 effective information, while the radar effective information remains at 0.09. The fused due east coordinates are firmly anchored to the radar end: Meters. Altitude fusion is 150.8 meters. Velocity fusion is 9.9 meters east, 5.1 meters south, and 0.05 meters per second ascent. The data achieves absolute mathematical self-consistency.

[0074] 205. Extract the optimal motion state estimate of the UAV from the fused state probability distribution, and predict and output the trajectory accordingly.

[0075] Specifically, the optimal state vector and the corresponding fusion covariance matrix at the current moment are obtained by parsing the fusion state probability distribution; a state transition matrix is ​​constructed based on a preset UAV dynamics model; the optimal state vector is used as the initial state, and multi-step iterative prediction is performed using the state transition matrix to obtain a sequence of predicted state vectors at multiple discrete time points in the future; at the same time, the fusion covariance matrix is ​​propagated through the state transition matrix and the process noise matrix to obtain the corresponding predicted covariance sequence; the predicted state vector sequence and the predicted covariance sequence are combined to generate and output the final predicted trajectory sequence.

[0076] It should be noted that the extreme value center is directly extracted from the fused distribution output in step 204. The optimal 3D position of the UAV at the current time T0 is absolutely anchored as follows: 1200.3 meters due east, 798.9 meters due north, and 150.8 meters altitude; the 3D velocity is 9.9 m / s eastward, -5.1 m / s northward, and 0.05 m / s vertically. Simultaneously, the highly convergent fused covariance matrix is ​​extracted, representing that the current positioning result has the highest level of confidence in the entire system.

[0077] The system utilizes a three-dimensional constant velocity (CV) dynamic model and a state transition matrix with a time step of 1 second. The system performs integration and forward propagation using the state at time T0 as the initial anchor point.

[0078] In the first second (T1), the position due east is updated to 1200.3 + 9.9 = 1210.2 meters, and the position due north is 798.9 - 5.1 = 793.8 meters. Subsequent nodes are calculated similarly. Simultaneously, the converged fusion covariance matrix is ​​propagated by multiplying it with the atmospheric process noise matrix through the state transition equation. As the prediction time increases, the error ellipsoid volume represented by the prediction covariance inevitably and smoothly amplifies.

[0079] The discrete predicted waypoints are structurally bound to the expanded error boundary to generate a rigorous predicted trajectory sequence as shown in Table 2 below, which is then sent to the fire interception or command and control network: Table 2 206. Before generating the fused state probability distribution, the radar state probability distribution and the visual state probability distribution are aligned in a common state space to obtain the radar subspace probability distribution and the visual subspace probability distribution, respectively. The common state space is the space composed of position and velocity components. The radar subspace probability distribution and the visual subspace probability distribution are used to obtain two corresponding manifold points on the statistical manifold.

[0080] Specifically, the radar state probability distribution and the visual state probability distribution are aligned to a common state space, including: extracting a first multidimensional state vector and a first covariance matrix from the radar state probability distribution; constructing a first projection transformation matrix according to the dimension definition of the common state space; performing dimensionality reduction projection on the first multidimensional state vector using the first projection transformation matrix to obtain the radar subspace state vector, and simultaneously transforming the first covariance matrix to obtain the radar subspace covariance matrix; combining the radar subspace state vector and the radar subspace covariance matrix to form the radar subspace probability distribution; and performing the same projection transformation operation on the visual state probability distribution based on a common state space of the same dimension to generate the visual subspace probability distribution.

[0081] It should be noted that, according to the inherent logic of the algorithm, this dimension alignment step is triggered before entering manifold mapping 203. Since radar has micro-Doppler characteristics and vision has pixel-size characteristics, this step aims to reduce the heterogeneous high-dimensional distribution through mathematical dimensionality reduction, uniformly pruning it into a standardized subspace that only includes the core kinematic parameters.

[0082] At the underlying level, an absolute 6-dimensional common state space (3D position + 3D velocity) is defined. For the 8-dimensional distribution of the radar, a 6x8 projection transformation matrix is ​​constructed (the left side is a 6th-order identity matrix, and the two right columns are all zeros) to accurately remove the micro-motion features of the latter two dimensions. Similarly, a matching projection matrix is ​​constructed for visual data to remove pixel features.

[0083] Perform a matrix projection operation to cut the 8-dimensional state vectors of radar and vision into a 6-dimensional subspace state vector of pure kinematics; simultaneously use projection transpose to reduce the 8-order covariance matrix to a 6-order principal submatrix, completely removing the variance interaction of the feature dimensions.

[0084] It is worth noting the logical consistency: the stripped radar micro-motion features (such as 60 Hz rotation speed) were not simply discarded, but were bypassed by the system and output to an independent target classification and recognition module, which is dedicated to determining the UAV model. This achieves both dimensional alignment of trajectory fusion and maximizes the use of computing power.

[0085] After rigorous alignment processing, the following standardized 6D data was successfully generated (see Table 3 below) for subsequent manifold fusion: Table 3 Figure 4This is a schematic diagram of a low-altitude UAV trajectory tracking device based on multi-source data according to an embodiment of the present invention. The low-altitude UAV trajectory tracking device 300 based on multi-source data can vary considerably due to differences in configuration or performance. The low-altitude UAV trajectory tracking device 300 based on multi-source data includes a transmitter 301, a receiver 302, and a processor 303. The processor 303 can also be a controller. Figure 4 This is referred to as "controller / processor 303". Optionally, the low-altitude UAV trajectory tracking device 300 based on multi-source data may also include a modem processor 305, wherein the modem processor 305 may include an encoder 306, a modulator 307, a decoder 308, and a demodulator 309.

[0086] In one example, transmitter 301 modulates (e.g., analog-to-analog conversion, filtering, amplification, and up-conversion, etc.) the output sample and generates an uplink signal that is transmitted via an antenna to an access network device. On the downlink, the antenna receives the downlink signal transmitted by the access network device. Receiver 302 modulates (e.g., filtering, amplification, down-conversion, and digitization, etc.) the signal received from the antenna and provides an input sample. In modem processor 305, encoder 306 receives traffic data and signaling messages to be transmitted on the uplink and processes (e.g., formatting, encoding, and interleaving) the traffic data and signaling messages. Modulator 307 further processes (e.g., symbol mapping and modulation) the encoded traffic data and signaling messages and provides an output sample. Demodulator 309 processes (e.g., demodulates) the input sample and provides a symbol estimate. Decoder 308 processes (e.g., deinterleaving and decoding) the symbol estimate and provides the decoded data and signaling messages to a low-altitude UAV trajectory tracking device 300 based on multi-source data. The encoder 306, modulator 307, demodulator 309, and decoder 308 can be implemented by a combined modem processor 305. These units process data according to the radio access technology used by the radio access network (e.g., LTE and other evolved systems access technologies). It should be noted that when the low-altitude UAV trajectory tracking device 300 based on multi-source data does not include the modem processor 305, the aforementioned functions of the modem processor 305 can also be performed by the processor 303.

[0087] The processor 303 controls and manages the actions of the low-altitude UAV trajectory tracking device 300 based on multi-source data, and is used to execute the processing procedures performed by the low-altitude UAV trajectory tracking device 300 based on multi-source data in the embodiments of this disclosure described above. For example, the processor 303 is also used to execute various steps of the transmitting or receiving device in the above method embodiments, and / or other steps of the technical solutions described in the embodiments of this disclosure.

[0088] Furthermore, the low-altitude UAV trajectory tracking device 300 based on multi-source data may also include a memory 304 for storing program code and data for the low-altitude UAV trajectory tracking device 300 based on multi-source data.

[0089] Understandable Figure 4 This illustration only shows a simplified design of a low-altitude UAV trajectory tracking device 300 based on multi-source data. In practical applications, the low-altitude UAV trajectory tracking device 300 based on multi-source data may include any number of transmitters, receivers, processors, modem processors, memory, etc., and all devices that can implement the embodiments of this disclosure are within the protection scope of this disclosure.

[0090] The present invention also provides a low-altitude unmanned aerial vehicle (UAV) trajectory tracking device based on multi-source data. The low-altitude UAV trajectory tracking device based on multi-source data includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor performs the steps of the low-altitude UAV trajectory prediction method based on multi-source data in the above embodiments.

[0091] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when the instructions are executed on a computer, cause the computer to perform the steps of the low-altitude UAV trajectory prediction method based on multi-source data.

[0092] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0093] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0094] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting the trajectory of a low-altitude unmanned aerial vehicle (UAV) based on multi-source data, characterized in that, include: Process observation data from the source phased array radar to generate radar state probability distribution; Process image sequences from visual sensors and generate visual state probability distributions through moving target detection and feature tracking; By mapping the radar state probability distribution and the visual state probability distribution to the same statistical manifold, two corresponding manifold points are obtained. The two manifold points are weighted based on sensor confidence, and the probability distribution of the fused state is obtained by minimizing the geometric distance on the manifold. The optimal motion state estimate of the UAV is extracted from the fused state probability distribution for trajectory prediction and output.

2. The low-altitude UAV trajectory prediction method based on multi-source data according to claim 1, characterized in that, include: Acquire observational data, including three-dimensional point clouds and micro-Doppler matrices, from a source phased array radar; Image morphological filtering and connected component analysis are performed on the micro-Doppler matrix to extract stable micro-motion features and generate radar feature vectors; By combining the position and velocity information of the three-dimensional point cloud with the radar feature vector, a radar state vector is constructed. Based on the radar state vector and the preset radar measurement noise model, a radar state probability distribution is generated.

3. The low-altitude UAV trajectory prediction method based on multi-source data according to claim 2, characterized in that, include: Acquire image sequences captured by the vision sensor; The image sequence is subjected to moving target detection based on optical flow calculation to obtain the target region including the UAV; Multiple image feature points are extracted within the target area, and the movement of the image feature points in the sequence is tracked to generate a set of two-dimensional feature point trajectories; Based on the camera imaging model, the set of two-dimensional feature point trajectories is back-projected to calculate the corresponding set of three-dimensional spatial point motion trajectories. Statistical analysis is performed on the set of motion trajectories of the three-dimensional spatial points to obtain a visual state vector representing the motion state of the UAV; Based on the visual state vector and the preset visual measurement error model, a visual state probability distribution is generated.

4. The low-altitude UAV trajectory prediction method based on multi-source data according to claim 3, characterized in that, include: Extract the radar state vector and radar uncertainty parameters from the radar state probability distribution; Extract the visual state vector and visual uncertainty parameter from the visual state probability distribution; The radar state vector and radar uncertainty parameters, as well as the visual state vector and visual uncertainty parameters, are respectively converted into coordinate representations on a statistical manifold; Based on the coordinate representation, corresponding radar manifold points and visual manifold points are determined on the statistical manifold.

5. The low-altitude UAV trajectory prediction method based on multi-source data according to claim 4, characterized in that, include: Based on the covariance matrix in the radar uncertainty parameters, the corresponding radar Fisher information matrix is ​​calculated; Based on the covariance matrix in the aforementioned visual uncertainty parameters, the corresponding visual Fisher information matrix is ​​calculated; The radar state vector is combined with the radar Fisher information matrix to form a complete geometric expression of the radar probability distribution, which serves as the coordinates of the radar manifold points. The visual state vector is combined with the visual Fisher information matrix to form a complete geometric expression of the visual probability distribution, which serves as the coordinates of the visual manifold points.

6. The low-altitude UAV trajectory prediction method based on multi-source data according to claim 4, characterized in that, include: The radar confidence weight and the visual confidence weight are dynamically determined based on the real-time measurement quality of the source phased array radar and the visual sensor. Based on the radar confidence weight and the visual confidence weight, the geometric representations of the radar manifold points and the visual manifold points on the statistical manifold are weighted to obtain the weighted manifold point representations. On the statistical manifold, a fusion point is found through optimization calculation that minimizes the combined geometric distance from the fusion point to the weighted radar manifold point and the visual manifold point, thus obtaining the fused manifold point; The fused manifold points are mapped back to the probability distribution parameter space to generate the fused state probability distribution.

7. The low-altitude UAV trajectory prediction method based on multi-source data according to claim 6, characterized in that, The global weights are calculated using the range normalization exponent formula: ; in, Environmental penalty index; For the real-time signal-to-noise ratio of the radar; This represents the minimum radar signal-to-noise ratio. This represents the maximum value of the radar signal-to-noise ratio; Variance of real-time visual sharpness; This represents the minimum value of the variance in visual acuity; This represents the maximum value of the variance in visual acuity.

8. The low-altitude UAV trajectory prediction method based on multi-source data according to claim 6, characterized in that, include: The optimal state vector and the corresponding fusion covariance matrix at the current moment are obtained by parsing the fusion state probability distribution. A state transition matrix is ​​constructed based on a pre-defined UAV dynamics model; Using the optimal state vector as the initial state, multi-step iterative prediction is performed using the state transition matrix to obtain a sequence of predicted state vectors for multiple discrete time points in the future. At the same time, the fusion covariance matrix is ​​propagated through the state transition matrix and the process noise matrix to obtain the corresponding predicted covariance sequence. The predicted state vector sequence is combined with the predicted covariance sequence to generate and output the predicted trajectory sequence.

9. The low-altitude UAV trajectory prediction method based on multi-source data according to claim 1, characterized in that, The radar state probability distribution and the visual state probability distribution are aligned in a common state space to obtain the radar subspace probability distribution and the visual subspace probability distribution, respectively. The common state space is a space composed of position and velocity components.

10. The low-altitude UAV trajectory prediction method based on multi-source data according to claim 9, characterized in that, Extract the first multidimensional state vector and the first covariance matrix from the radar state probability distribution; Based on the definition of the dimension of the common state space, construct the first projection transformation matrix; The first multidimensional state vector is reduced in dimension by using the first projection transformation matrix to obtain the radar subspace state vector, and the first covariance matrix is ​​transformed simultaneously to obtain the radar subspace covariance matrix. The radar subspace state vector and the radar subspace covariance matrix are combined to form the radar subspace probability distribution; Using the common state space of the same dimension as a reference, the same projection transformation operation is performed on the visual state probability distribution to generate a visual subspace probability distribution.

Images