Aircraft trajectory fitting method and device based on multi-modal information, and storage medium

Abstract

Embodiments of the present application provide a kind of aircraft trajectory fitting method, equipment and storage medium based on multi-modal information.The method comprises: obtaining the flight data of different sources of flying body in current airspace;Flight data is filtered, and the flight data not belonging to aircraft is filtered;Based on the flight data of different sources of aircraft, the motion feature matrix of corresponding source is constructed, and the flight data of different sources of aircraft is aligned in time and space;According to motion feature matrix, the flight data of aligned aircraft is evaluated according to feature matching degree, and the flight data set of aircraft belonging to the same aircraft is obtained;According to the flight data set of aircraft belonging to the same aircraft, the flight trajectory of aircraft is determined.The method is used to achieve that the difference between the unmanned aerial vehicle flight trajectory fitted and the real flight trajectory of unmanned aerial vehicle is smaller, improve unmanned aerial vehicle flight trajectory fitting effect, and then the monitoring effect of unmanned aerial vehicle can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a flight vehicle trajectory fitting method based on multi-modal information, a device and a storage medium. BACKGROUND

[0002] In recent years, the unmanned aerial vehicle technology has made remarkable progress. In flight control, navigation positioning, load carrying, communication transmission and other aspects, it is constantly optimized and deeply integrated with 5G, artificial intelligence and other technologies, further expanding its application scenarios and market space. Unmanned aerial vehicles have a wide range of application scenarios and a broad market space. Their industry applications are becoming more diverse and in-depth, and the related supporting development is also becoming more and more perfect.

[0003] In the related art, in order to monitor the unmanned aerial vehicle in real time, the flight trajectory of the unmanned aerial vehicle needs to be fitted, and the unmanned aerial vehicle is monitored in real time through the fitted unmanned aerial vehicle trajectory. However, the flight data of the existing unmanned aerial vehicle is collected only by one of the sensors and the radar, and there is a large difference between the fitted unmanned aerial vehicle flight trajectory and the real flight trajectory of the unmanned aerial vehicle. The fitting effect of the unmanned aerial vehicle flight trajectory is poor, which leads to poor monitoring effect of the unmanned aerial vehicle.

[0004] Therefore, in order to improve the fitting effect of the unmanned aerial vehicle flight trajectory, a method for fitting the trajectory of an unmanned aerial vehicle based on multi-modal information is needed. SUMMARY

[0005] The embodiments of the present application provide a flight vehicle trajectory fitting method based on multi-modal information, a device and a storage medium, to solve the problem of poor fitting effect of the unmanned aerial vehicle flight trajectory.

[0006] In a first aspect, the embodiments of the present application provide a flight vehicle trajectory fitting method based on multi-modal information, comprising:

[0007] Obtaining flight data of flight bodies of different sources in a current airspace, different sources being used to represent different collection methods, different collection methods corresponding to different modalities; wherein the flight bodies include at least one moving object in the air;

[0008] Filtering flight data that does not belong to the flight vehicle, to obtain flight vehicle flight data;

[0009] Based on the flight vehicle flight data of different sources, constructing a motion feature matrix corresponding to the source, and aligning the flight vehicle flight data of different sources in time and space to obtain aligned flight vehicle flight data;

[0010] According to the motion feature matrix, evaluating the feature matching degree of the aligned flight vehicle flight data to obtain a flight vehicle flight data set belonging to the same flight vehicle;

[0011] determine the flight trajectory of the aircraft according to the aircraft flight data set belonging to the same aircraft.

[0012] In a possible implementation, the flight data includes first aircraft data and flight body data of all flight bodies in the current airspace; the flight data of flight bodies of different sources in the current airspace is acquired, including:

[0013] The first aircraft data of the aircraft itself is acquired by a sensor arranged on the aircraft, and the first aircraft data includes first aircraft positioning data, first aircraft identification, and first aircraft time data.

[0014] The flight body data of all flight bodies in the current airspace is acquired by an airspace management center arranged on the ground, wherein the flight body data includes flight body time data, flight body positioning data, flight body speed data, and flight body angle data; and the airspace management center at least includes a plurality of 5G-A base stations, an ADS-B system, an integrated fusion ground monitoring base station, different types of sensors, and a radar system, and the integrated fusion ground monitoring base station is used to represent a ground base station with an aircraft monitoring function.

[0015] In a possible implementation, the flight data is screened, and flight data not belonging to the aircraft is filtered to obtain aircraft flight data, including:

[0016] The current airspace is divided into a plurality of sub-airspaces, and a geographical position code list corresponding to each sub-airspace is generated;

[0017] Each flight body positioning data is inserted into a spatial index, and a trajectory bounding box corresponding to the flight body positioning data is obtained in combination with the geographical position code list;

[0018] The relevance between different trajectory bounding boxes is calculated;

[0019] Flight body flight data not belonging to the aircraft is determined according to the relevance;

[0020] The flight body flight data not belonging to the aircraft is filtered to obtain second aircraft data, and the second aircraft data and the first aircraft data are combined to obtain aircraft flight data, wherein the second aircraft data includes second aircraft time data, second aircraft positioning data, second aircraft speed data, and second aircraft angle data.

[0021] In a possible implementation, the motion feature matrix includes average speed, speed variation coefficient, and acceleration spectrum.

[0022] Based on the aircraft flight data of different sources, a motion feature matrix corresponding to the source is constructed, including:

[0023] The first average speed, the first speed variation coefficient and the first acceleration spectrum are calculated based on the first aircraft positioning data and the first aircraft time data, and the first motion feature matrix is constructed.

[0024] The second average speed, the second speed variation coefficient and the second acceleration spectrum are calculated based on the second aircraft time data, the second aircraft positioning data, the second aircraft speed data and the second aircraft angle data, and the second motion feature matrix is constructed.

[0025] In a possible implementation, the aircraft flight data of different sources are aligned in time and space to obtain aligned aircraft flight data, including:

[0026] The first aircraft time data and the second aircraft time data are time-aligned to obtain time-aligned first aircraft data and time-aligned second aircraft data;

[0027] The cosine similarity between the first motion feature matrix and the second motion feature matrix is calculated, the first spatial rotation matrix is constructed according to the cosine similarity, the first aircraft flight data and the second aircraft flight data are spatially aligned through the first spatial rotation matrix to obtain aligned first aircraft data and aligned second aircraft data, and the aligned first aircraft data and the aligned second aircraft data are taken as the aligned aircraft flight data.

[0028] In a possible implementation, the aligned aircraft flight data is evaluated according to the motion feature matrix to obtain a set of aircraft flight data belonging to the same aircraft, including:

[0029] The first motion feature matrix and the second motion feature matrix are input into a pre-trained random forest model, and the feature matching degree between the first motion feature matrix and the second motion feature matrix is output;

[0030] The first motion feature matrix and the second motion feature matrix with the feature matching degree greater than the feature matching degree threshold are screened, it is determined that the first aircraft flight data corresponding to the first motion feature matrix and the second aircraft flight data corresponding to the second motion feature matrix with the feature matching degree greater than the feature matching degree threshold belong to the aircraft flight data of the same aircraft, and the first aircraft flight data and the second aircraft flight data belonging to the same aircraft are combined to obtain a set of aircraft flight data belonging to the same aircraft.

[0031] In a possible implementation, the flight trajectory of the aircraft is determined according to the set of aircraft flight data belonging to the same aircraft, including:

[0032] set hash indexes of spatiotemporal cubes corresponding to the first aircraft flight data and the second aircraft flight data belonging to the same aircraft respectively, based on the hash indexes, deduplicate the first aircraft flight data and the second aircraft flight data belonging to the same aircraft to obtain deduplicated aircraft flight data at multiple time points;

[0033] adopt the Kalman filtering algorithm to filter the deduplicated aircraft flight data at each time point to determine target aircraft flight data;

[0034] when the target aircraft flight data at the current time point is lacking, adopt the cubic spline interpolation to fill in the target aircraft flight data at the current time point until the target aircraft flight data at each time point exists, combine the target aircraft flight data at each time point to obtain fused aircraft flight data, and determine the flight trajectory of the aircraft according to the fused aircraft flight data.

[0035] In a possible implementation, before determining the flight trajectory of the aircraft according to the set of aircraft flight data belonging to the same aircraft, the method further includes:

[0036] calculate the mean and standard deviation of the aircraft flight data belonging to the same aircraft from different sources by the Isolation Forest algorithm;

[0037] construct a threshold range based on the mean and standard deviation, determine aircraft flight data not belonging to the threshold range as an abnormal point, perform abnormal point cleaning on the aircraft flight data belonging to the same aircraft to obtain cleaned aircraft flight data, and determine the flight trajectory of the aircraft according to the cleaned aircraft flight data.

[0038] In a second aspect, the embodiments of the present application provide an aircraft trajectory fitting device based on multi-modal information, comprising:

[0039] a data acquisition module configured to acquire flight data of flight bodies from different sources in a current airspace, wherein different sources represent different collection methods, different collection methods correspond to different modalities, and the flight bodies include at least one moving object in the air;

[0040] a data screening module configured to screen the flight data to filter flight data not belonging to an aircraft to obtain aircraft flight data;

[0041] a feature matrix construction module configured to construct a motion feature matrix corresponding to a source based on aircraft flight data from different sources, and align the aircraft flight data from different sources in time and space to obtain aligned aircraft flight data;

[0042] The feature matching degree evaluation module is used to evaluate the feature matching degree of the aligned aircraft flight data based on the motion feature matrix, so as to obtain a set of aircraft flight data belonging to the same aircraft.

[0043] The trajectory determination module is used to determine the flight trajectory of an aircraft based on a set of flight data belonging to the same aircraft.

[0044] In one possible implementation, the flight data includes data from the first aircraft and flight data from all aircraft within the current airspace; the data acquisition module is specifically used for:

[0045] The first aircraft data is collected by sensors installed on the aircraft. The first aircraft data includes: first aircraft positioning data, first aircraft identification and first aircraft time data.

[0046] By setting up an airspace management center on the ground, flight data of all flying objects in the current airspace is collected. The flight data includes: flight time data, flight position data, flight speed data, and flight angle data. The airspace management center includes at least: several 5G-A base stations, an ADS-B system, an integrated ground surveillance base station, and different types of sensors and radar systems. The integrated ground surveillance base station is used to represent ground base stations with aircraft surveillance capabilities.

[0047] In one possible implementation, the data filtering module is specifically used for:

[0048] Divide the current airspace into several sub-airspaces and generate a list of geolocation codes for each sub-airspace.

[0049] Insert the positioning data of each flying object into the spatial index, and combine it with the geolocation code list to obtain the trajectory bounding box corresponding to the positioning data of the flying object;

[0050] Calculate the correlation between different trajectory bounding boxes;

[0051] Based on correlation, flight data of flight bodies that do not belong to aircraft were identified;

[0052] Flight data of non-aircraft objects is filtered to obtain second-aircraft data. This second-aircraft data is then combined with the first-aircraft data to obtain the final aircraft flight data. The second-aircraft data includes: second-aircraft time data, second-aircraft positioning data, second-aircraft velocity data, and second-aircraft angle data. In one possible implementation, the motion feature matrix includes average velocity, velocity variation coefficient, and acceleration spectrum.

[0053] The feature matrix construction module is specifically used for:

[0054] Based on the positioning data and time data of the first aircraft, the first average velocity, the first velocity variation coefficient and the first acceleration spectrum are calculated, and the first motion feature matrix is ​​constructed.

[0055] Based on the second aircraft's time data, second aircraft's positioning data, second aircraft's velocity data, and second aircraft's angle data, the second average velocity, the second velocity variation coefficient, and the second acceleration spectrum are calculated, and the second motion feature matrix is ​​constructed.

[0056] In one possible implementation, the feature matrix construction module is also specifically used for:

[0057] Time alignment is performed on the time data of the first and second aircraft to obtain time-aligned data of the first and second aircraft.

[0058] Calculate the cosine similarity between the first motion feature matrix and the second motion feature matrix. Based on the cosine similarity, construct a first spatial rotation matrix. Using the first spatial rotation matrix, spatially align the first and second aircraft flight data to obtain aligned first and second aircraft data. Use the aligned first and second aircraft data as the aligned aircraft data.

[0059] In one possible implementation, the feature matching degree evaluation module is specifically used for:

[0060] Input the first motion feature matrix and the second motion feature matrix into a pre-trained random forest model, and output the feature matching degree between the first motion feature matrix and the second motion feature matrix;

[0061] First motion feature matrices and second motion feature matrices with feature matching degrees greater than feature matching degree thresholds are selected. The first aircraft flight data corresponding to the first motion feature matrix with feature matching degree greater than feature matching degree threshold and the second aircraft flight data corresponding to the second motion feature matrix with feature matching degree greater than feature matching degree threshold are determined to be aircraft flight data belonging to the same aircraft. The first aircraft flight data and the second aircraft flight data belonging to the same aircraft are combined to obtain the set of aircraft flight data belonging to the same aircraft.

[0062] In one possible implementation, the trajectory determination module is specifically used for:

[0063] Set hash indices for the spatiotemporal cubes corresponding to the first and second aircraft flight data belonging to the same aircraft, respectively. Based on the hash indices, deduplicate the first and second aircraft flight data belonging to the same aircraft to obtain deduplicated aircraft flight data at multiple time points.

[0064] The Kalman filter algorithm is used to filter the deduplicated flight data of the aircraft at each time point to determine the flight data of the target aircraft.

[0065] When there is a lack of target aircraft flight data at the current time point, cubic spline interpolation is used to fill in the target aircraft flight data at the current time point until target aircraft flight data exists at each time point. The target aircraft flight data at each time point are combined to obtain fused aircraft flight data. Based on the fused aircraft flight data, the flight trajectory of the aircraft is determined.

[0066] In one possible implementation, the aircraft trajectory fitting device based on multimodal information is also specifically used for:

[0067] The isolated forest algorithm is used to calculate the mean and standard deviation of flight data of the same aircraft from different sources.

[0068] A threshold range is constructed based on the mean and standard deviation. Flight data of aircraft that do not fall within the threshold range are identified as outliers. Outlier cleaning is performed on flight data of aircraft belonging to the same aircraft to obtain cleaned flight data. The flight trajectory of the aircraft is then determined based on the cleaned flight data.

[0069] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor;

[0070] The memory stores instructions that the computer executes;

[0071] The processor executes computer execution instructions stored in memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0072] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible implementations of the first aspect.

[0073] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the first aspect and / or various possible implementations of the first aspect.

[0074] The aircraft trajectory fitting method, device, and storage medium based on multimodal information provided in this application acquire flight data of different modes of aircraft collected by various acquisition methods. This multimodal flight data can more comprehensively and accurately characterize the flight of the aircraft, facilitating subsequent trajectory fitting. By filtering out flight data that does not belong to the aircraft, the amount of flight data required for trajectory fitting is reduced, further improving the speed and accuracy of trajectory fitting. Spatiotemporal alignment of flight data from different sources standardizes the flight data of different modes to the same spatiotemporal context, avoiding inaccuracies in flight data fitting due to inconsistencies between different modes. A motion feature matrix is ​​used to distinguish flight data sets belonging to the same aircraft from those of different aircraft, facilitating subsequent trajectory fitting using data belonging to the same aircraft. By fusing the flight data of the same aircraft, the flight trajectory of the aircraft is determined. The difference between the fitted UAV flight trajectory and the actual flight trajectory of the UAV is small, achieved by using flight data of different modes collected by various acquisition methods, thus improving the UAV flight trajectory fitting effect and consequently enhancing the monitoring effect of the UAV. Attached Figure Description

[0075] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0076] Figure 1 A schematic diagram of the scenario for fitting the flight trajectory of the UAV provided in this application;

[0077] Figure 2 A flowchart illustrating the aircraft trajectory fitting method based on multimodal information provided in this application;

[0078] Figure 3 A schematic diagram of the structure of the aircraft trajectory fitting device based on multimodal information provided in this application;

[0079] Figure 4 A schematic diagram of the structure of the electronic device provided in this application.

[0080] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0081] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0082] First, let me explain the terms used in this application:

[0083] 5G-A (5G-Advanced, fifth-generation mobile communication technology evolution): is an evolution and enhancement of the 5G network in terms of functionality and coverage; 5G-A base stations are base stations that support the 5G-A network and have integrated sensing and communication (ISAC) capabilities. 5G-A base stations can not only transmit data, but also realize environmental perception (such as vehicle detection, drone tracking, etc.).

[0084] ADS-B (Automatic Dependent Surveillance-Broadcast): refers to air-to-air traffic surveillance between aircraft with the ability to broadcast position reports.

[0085] Integrated ground surveillance base station: refers to a manned / unmanned integrated ground surveillance base station. This base station can capture communication data between the drone and the remote controller, receive and parse the drone Remote ID broadcast signal that conforms to multiple international standards, as well as the aircraft's ADS-B message, thereby obtaining and updating key information such as the identity information, geographical location, and flight status of manned and unmanned aircraft in real time.

[0086] Figure 1 A schematic diagram of the scenario for fitting the flight trajectory of the UAV provided in this application, such as... Figure 1 As shown, the specific application scenario of this application is for monitoring drones. Monitoring drones may include: collecting drone flight data through acquisition device 101 and sending the flight data to computing device 102; computing device 102 using the drone flight data to fit the drone's flight trajectory to obtain the drone trajectory, and monitoring the drone in real time based on the fitted drone trajectory.

[0087] Based on the above scenarios, it can be seen that in the existing technology, the flight data of drones is collected only through one of the methods of sensors or radar. There is a large difference between the fitted drone flight trajectory and the actual flight trajectory of the drone. The drone flight trajectory fitting effect is poor, resulting in poor monitoring effect of drones.

[0088] The aircraft trajectory fitting method based on multimodal information provided in this application acquires flight data of different modes of aircraft collected through various acquisition methods. This multimodal flight data provides a more comprehensive and accurate representation of the aircraft's flight, facilitating subsequent trajectory fitting. By filtering out flight data that does not belong to the aircraft, the required data for trajectory fitting is reduced, further improving the speed and accuracy of trajectory fitting. Spatiotemporal alignment of flight data from different sources standardizes the data to the same spatiotemporal context, avoiding inconsistencies that could lead to inaccurate fitting. A motion feature matrix distinguishes flight data belonging to the same aircraft from data belonging to different aircraft, facilitating trajectory fitting using this data. Finally, by fusing the data from the same aircraft, the flight trajectory is determined. The fitted UAV flight trajectory obtained from the multimodal flight data acquired through various acquisition methods shows minimal difference from the actual UAV flight trajectory, improving the trajectory fitting effect and thus enhancing UAV monitoring.

[0089] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0090] Figure 2 A flowchart illustrating the aircraft trajectory fitting method based on multimodal information provided in this application is shown below. Figure 2 As shown, the method includes:

[0091] S201. Obtain flight data of aircraft from different sources within the current airspace.

[0092] In one possible implementation, different sources are used to characterize different acquisition methods, and different acquisition methods correspond to different modes; the flying body includes at least one moving object in the air.

[0093] In some embodiments, flight data includes data of a first aircraft and flight data of all aircraft within the current airspace; acquiring flight data of aircraft from different sources within the current airspace includes:

[0094] The first aircraft data is collected by sensors installed on the aircraft. The first aircraft data includes: first aircraft positioning data, first aircraft identification and first aircraft time data.

[0095] By setting up an airspace management center on the ground, flight data of all flying objects in the current airspace is collected. The flight data includes: flight time data, flight position data, flight speed data, and flight angle data. The airspace management center includes at least: several 5G-A base stations, an ADS-B system, an integrated ground surveillance base station, and different types of sensors and radar systems. The integrated ground surveillance base station is used to represent ground base stations with aircraft surveillance capabilities.

[0096] In one example, each airspace has a longitude range, a latitude range, and an altitude range. The positioning data of the first aircraft may include the longitude, latitude, and altitude of the aircraft. The first aircraft identifier is obtained through the flight control system set up on the aircraft. The longitude, latitude, and altitude of the aircraft are measured. Based on the longitude, latitude, and altitude of the aircraft, the positioning data of the first aircraft is obtained. It is determined whether the longitude, latitude, and altitude of the aircraft are within the longitude, latitude, and altitude range of the current airspace. If the longitude, latitude, and altitude of the aircraft are within the longitude, latitude, and altitude range of the current airspace, it is determined that the aircraft belongs to the current airspace. The positioning data of the first aircraft is timestamped through the flight control system set up on the aircraft to obtain the time data of the first aircraft.

[0097] Optionally, flight object data can be obtained through 5G-A base stations: 5G-A base stations transmit wireless signals for sensing, where the wireless signals are reflected, scattered, or diffracted when they encounter a flight object in the current airspace, resulting in a feedback signal containing the spatial and motion information of the flight object; 5G-A base stations process the feedback signal using signal processing algorithms to obtain flight object data, which may include: obtaining flight object time data by using the timestamps of the wireless signal transmission and reception of the feedback signal; 5G-A base stations obtain the distance and angle of arrival of the flight object based on the feedback signal, and calculate the position of the flight object in three-dimensional space by combining the 5G-A base station's own geographical location information, obtaining flight object positioning data; 5G-A base stations obtain the longitudinal velocity of the flight object through Doppler frequency shift, and decompose the radial velocity into a three-dimensional coordinate system by combining the continuous position change difference of the flight object in three-dimensional space with the angle of arrival, obtaining flight object flight speed data; 5G-A base stations calculate the heading angle data of the flight object by using the direction of the flight object's velocity vector, obtaining flight object angle data.

[0098] Optionally, flight data can be acquired through the ADS-B system: The ADS-B system includes a global navigation satellite system and an inertial navigation system. High-precision UTC (Coordinated Universal Time) provided by the global navigation satellite system provides flight time data; the global navigation satellite system, combined with the inertial navigation system, calculates the flight's three-dimensional coordinates in real time to obtain flight positioning data; the ground velocity and trajectory heading angle provided by the global navigation satellite system are used to calculate flight velocity through vector synthesis and dynamic pressure; and the trajectory heading angle is corrected by the global navigation satellite system to obtain flight angle data.

[0099] Optionally, flight data can be obtained through an integrated ground surveillance base station: the integrated ground surveillance base station must have at least radio spectrum detection capability, Remote ID (Remote Identification) broadcast signal reception and parsing capability, and ADS-B message reception and parsing capability. Through the radio spectrum detection capability and Remote ID broadcast signal reception and parsing capability of the integrated ground surveillance base station, data such as latitude, longitude, altitude, and speed of the flight body can be obtained to obtain the flight body's positioning data and speed data; through the ADS-B message reception and parsing capability of the integrated ground surveillance base station, the aircraft's ADS-B messages can be received and parsed to obtain the flight time data and flight angle data of the flight body.

[0100] Optionally, flight data can be acquired using different types of sensors: the sensors include at least a global navigation satellite system and an inertial navigation system. The flight time data is obtained by using the satellite navigation system for atomic clock timing; the flight positioning data is obtained by using the satellite navigation system in conjunction with a barometric altimeter to acquire data such as the flight's longitude, latitude, and geometric altitude; the flight velocity data is calculated by using the ground velocity and trajectory angle acquired by the satellite navigation system in conjunction with the airspeed measured by the pitot tube sensor; and the flight angle data is obtained by using the inertial navigation system to measure the flight's heading data and using airflow sensors for auxiliary correction.

[0101] Optionally, flight data can be acquired through a radar system: using a high-precision clock (such as a rubidium atomic clock or a temperature-controlled crystal oscillator) inside the radar as a time reference, the flight time data is calculated by measuring the time difference between the transmission and reception of the radar electromagnetic waves; the flight angle data is determined by using a mechanically scanned radar to determine the antenna pointing angle when the echo is strongest, or by using a phase shifter of a phased array radar to control the beam pointing (electronic scanning) and output the flight angle data in real time; the radar system measures the distance and altitude of the flight, and combined with the flight angle data, converts it to a geodetic coordinate system to obtain the flight location data; the radar system calculates the radial velocity using the frequency offset of the echo signal, and combines it with differential velocimetry to obtain the flight velocity data.

[0102] In this embodiment, the first aircraft data is collected by sensors installed on the aircraft itself; the flight data of all flying objects in the airspace is collected by different types of airspace management centers set up on the ground. The flight data is more comprehensive and has more modalities, which improves the effect of fitting the UAV flight trajectory using flight data collected by multiple acquisition methods, and further improves the data quality and the monitoring effect of the UAV.

[0103] S202. Filter the flight data to obtain the flight data of the aircraft.

[0104] In one possible implementation, the flight data is filtered to remove data that does not belong to the aircraft, resulting in aircraft flight data, including:

[0105] Divide the current airspace into several sub-airspaces and generate a list of geolocation codes for each sub-airspace.

[0106] Insert the positioning data of each flying object into the spatial index, and combine it with the geolocation code list to obtain the trajectory bounding box corresponding to the positioning data of the flying object;

[0107] Calculate the correlation between different trajectory bounding boxes;

[0108] Based on correlation, flight data of flight bodies that do not belong to aircraft were identified;

[0109] The flight data of the aircraft that does not belong to the aircraft is filtered to obtain the second aircraft data. The second aircraft data and the first aircraft data are combined to obtain the aircraft flight data. The second aircraft data includes: second aircraft time data, second aircraft positioning data, second aircraft speed data and second aircraft angle data.

[0110] In one example, the Geohash (geocode system) grid system can be used to divide the current airspace into several sub-airspaces, encode the geographic location of each sub-airspace, and combine them to generate a list of geographic location codes corresponding to each sub-airspace in the current airspace.

[0111] The spatial index is inserted with the flight object's location data from the initial moment of detection in the current airspace to the current moment, and from all moments acquired using the current acquisition method. Each leaf node in the spatial index stores the flight object's location data corresponding to that leaf node. The root node automatically calculates and stores the union of the Minimum Bounding Rectangles (MBRs) of all leaf nodes under it. Specifically, starting from the root node of the spatial index, the index is accessed layer by layer: if the current node is a leaf node, the MBR of the flight object's location data for that node is calculated and stored; if the current node is not a leaf node, its child nodes are accessed until the current node becomes a leaf node, where the MBR of the flight object's location data is calculated and stored. The union of the MBRs of all leaf nodes is then used to obtain the trajectory bounding box corresponding to the trajectory obtained by the current acquisition method. All sub-airspaces covered by the trajectory bounding box are calculated, generating a set of geolocation codes corresponding to the trajectory bounding box coverage. The process involves calculating the bounding boxes of all flight objects within the current airspace, obtained through different acquisition methods, and their corresponding geolocation codes. The set similarity between the geolocation code sets corresponding to each pair of bounding boxes is then calculated, determining the set similarity as the correlation between the bounding boxes corresponding to the geolocation code sets. The set similarity between Geohash prefix sets can be calculated using methods such as the Jaccard similarity coefficient, overlap coefficient, and Dice coefficient. This application does not impose any restrictions on the calculation method for set similarity.

[0112] Because the motion characteristics of aircraft and non-aircraft flying bodies are not the same, the correlation between aircraft data and other aircraft data is higher than the correlation threshold, the correlation between aircraft data and non-aircraft flying body data is lower than the correlation threshold, and the correlation between non-aircraft flying body data and non-aircraft flying body data is lower than the correlation threshold.

[0113] Optionally, determining flight data of aircraft that do not belong to aircraft based on correlation can be achieved through the following steps: Obtain all correlation sets corresponding to each flight data set; if the number of correlations greater than a correlation threshold in the correlation set corresponding to the current flight data is less than or equal to a quantity threshold, then the current flight data is determined to be non-aircraft, and the current flight data is filtered; if the number of correlations greater than a correlation threshold in the correlation set corresponding to the current flight data is greater than a quantity threshold, then the current flight data is determined to be a suspected aircraft. The quantity threshold can be calculated using C(n,2) = n! / (2! (n-2)!), where n is the number of types of airspace management centers located on the ground.

[0114] Based on a pre-constructed set of aircraft flight trajectories, features of the bounding boxes corresponding to these trajectories are extracted. The pre-constructed set of aircraft flight trajectories conforms to an aircraft behavior model. A bounding box classifier is trained using these features. The trained bounding box classifier is then used to classify and identify the bounding boxes of suspected aircraft trajectories, determining whether the bounding box belongs to an aircraft. A trained threshold classifier is capable of classifying and identifying flight data of aircraft. If the bounding box of a suspected aircraft does not belong to an aircraft, the flight data of that suspected aircraft's bounding box is filtered out. If the bounding box belongs to an aircraft, the suspected aircraft is identified as an aircraft, and secondary aircraft data is obtained. This secondary aircraft data includes: secondary aircraft time data, secondary aircraft positioning data, secondary aircraft velocity data, and secondary aircraft angle data.

[0115] In this embodiment, spatial indexing is used for prior bounding box collision detection to quickly exclude flight data that does not belong to the aircraft, retaining only the flight body data belonging to the aircraft for subsequent UAV trajectory fitting. This reduces the computational load of flight data and further improves the accuracy and fitting effect of flight trajectory fitting. Geographic location encoding further compresses spatial information and reduces computational complexity.

[0116] S203. Based on the aircraft flight data from different sources, construct the motion feature matrix corresponding to the source, and align the aircraft flight data from different sources in time and space to obtain aligned aircraft flight data.

[0117] In one possible implementation, the motion characteristic matrix includes average velocity, velocity variation coefficient, and acceleration spectrum;

[0118] Based on aircraft flight data from different sources, motion feature matrices for each source are constructed, including:

[0119] Based on the positioning data and time data of the first aircraft, the first average velocity, the first velocity variation coefficient and the first acceleration spectrum are calculated, and the first motion feature matrix is ​​constructed.

[0120] Based on the second aircraft's time data, second aircraft's positioning data, second aircraft's velocity data, and second aircraft's angle data, the second average velocity, the second velocity variation coefficient, and the second acceleration spectrum are calculated, and the second motion feature matrix is ​​constructed.

[0121] In one example, the first aircraft positioning data includes the initial positioning data of the aircraft entering the current airspace and the aircraft's current real-time positioning data. The first aircraft time data includes the initial time data of the aircraft entering the current airspace and the aircraft's current real-time time data. A first distance is calculated between the aircraft's current real-time positioning data and the initial positioning data of the aircraft entering the current airspace. A first total time is calculated between the aircraft's current real-time time data and the initial time data of the aircraft entering the current airspace. The first total distance is divided by the first total time to obtain a first average velocity. A first velocity standard deviation is calculated based on the first average velocity. The ratio between the first velocity standard deviation and the first average velocity is calculated to obtain a first velocity variation coefficient. A first instantaneous velocity sequence is calculated based on the first aircraft positioning data and the first aircraft time data. A first instantaneous acceleration sequence is calculated based on the first instantaneous velocity sequence. A first acceleration spectrum is constructed by combining the first instantaneous velocity sequence and the first instantaneous acceleration sequence. The first average velocity, the first velocity standard deviation, and the first acceleration spectrum are combined to obtain a first motion feature matrix.

[0122] The second motion feature matrix is ​​used to characterize the second motion feature matrices calculated from the second aircraft flight data acquired through different acquisition methods. The second aircraft positioning data includes the second initial positioning data upon entering the current airspace and the aircraft's current second real-time positioning data. The second aircraft time data includes the second initial time data upon entering the current airspace and the aircraft's current second real-time time data. For each acquisition method, the second motion feature matrix corresponding to each acquisition method is calculated step-by-step as follows: Calculate the second distance between the aircraft's current second real-time positioning data and the second initial positioning data upon entering the current airspace; calculate the second total time between the aircraft's current second real-time time data and the second initial time data upon entering the current airspace; divide the second total distance by the second total time to obtain the second average speed. Calculate the second speed standard deviation based on the second average speed; calculate the ratio between the second speed standard deviation and the second average speed to obtain the second speed variation coefficient. Calculate the second speed standard deviation based on the difference between the second aircraft speed data and the second average speed at different times; calculate the ratio between the second speed standard deviation and the second average speed to obtain the second speed variation coefficient. The second acceleration sequence is obtained by differentiating the second aircraft velocity data and the second aircraft time data. The second acceleration spectrum is constructed by combining the second angle data. The second average velocity, the second velocity standard deviation, and the second acceleration spectrum are combined to obtain the second motion feature matrix.

[0123] In this embodiment, a first motion feature matrix is ​​constructed based on the first flight data collected by the aircraft itself to characterize the motion characteristics of the first flight data collected by the aircraft itself. A second motion feature matrix is ​​constructed corresponding to each type of second flight data collected by a ground-based airspace management center to characterize the motion characteristics of the second flight data collected by the ground-based airspace management center, thus achieving a unified expression of multimodal motion characteristics. Since the motion feature matrix is ​​used to characterize the aircraft's motion characteristics, it does not change after flight data transformation, facilitating subsequent feature evaluation using the motion feature matrix.

[0124] In one possible implementation, aircraft flight data from different sources are aligned in time and space to obtain aligned aircraft flight data, including:

[0125] Time alignment is performed on the time data of the first and second aircraft to obtain time-aligned data of the first and second aircraft.

[0126] Calculate the cosine similarity between the first motion feature matrix and the second motion feature matrix. Based on the cosine similarity, construct a first spatial rotation matrix. Using the first spatial rotation matrix, spatially align the first and second aircraft flight data to obtain aligned first and second aircraft data. Use the aligned first and second aircraft data as the aligned aircraft data.

[0127] For the second spacecraft time data acquired using different acquisition methods, the time data acquired by different methods are aligned pairwise using the following steps: The Dynamic Time Warping (DTW) algorithm is used to align the second spacecraft time data acquired by different acquisition methods to the same time domain. The DTW algorithm is then used to align the first spacecraft time data with the aligned second spacecraft time data to the same time domain, resulting in time-aligned first spacecraft data and time-aligned second spacecraft data.

[0128] For second-aircraft data acquired using different acquisition methods, the following steps are used to spatially align the data from different acquisition methods pairwise: Obtain the velocity vectors of the second motion feature matrix acquired using the current acquisition method and the second motion feature matrix acquired using another acquisition method at the same time point. Calculate the cosine similarity between the velocity vectors from the current acquisition method and the other acquisition method at the same time point. Construct a second spatial rotation alignment matrix to align the second-aircraft flight data from the current acquisition method with the second-aircraft flight data from the other acquisition method to the same airspace. This second spatial rotation matrix represents the rotation matrix with the highest cosine similarity to the second motion feature matrix across all second-aircraft flight time points. After aligning all second-vehicle flight data to the same airspace, the first-vehicle flight data and the aligned second-vehicle flight data are aligned to the same airspace through the following steps: Obtain the velocity vectors of the first motion feature matrix and the aligned second motion feature matrix at the current flight time point; calculate the cosine similarity between the velocity vectors of the first and second motion feature matrices at the current flight time point. Construct a first spatial rotation matrix, and use this first spatial rotation matrix to align the first-vehicle flight data and the aligned second-vehicle flight data to the same airspace. The first spatial rotation matrix represents the rotation matrix that maximizes the cosine similarity between the first and second motion feature matrices across all flight time points.

[0129] In this embodiment, the trajectory data of different aircraft are inconsistent in time series due to various reasons. By using the DTW algorithm to align the time series and using the cosine similarity of the motion feature matrix for spatial alignment, the aircraft data acquired by different acquisition methods can be aligned, enabling accurate comparison and analysis of the relationship between the aircraft flight data acquired by different acquisition methods.

[0130] S204. Based on the motion feature matrix, evaluate the feature matching degree of the aligned aircraft flight data to obtain a set of aircraft flight data belonging to the same aircraft.

[0131] In one possible implementation, based on the motion feature matrix, the aligned aircraft flight data are evaluated for feature matching degree to obtain a set of aircraft flight data belonging to the same aircraft, including:

[0132] Input the first motion feature matrix and the second motion feature matrix into a pre-trained random forest model, and output the feature matching degree between the first motion feature matrix and the second motion feature matrix;

[0133] First motion feature matrices and second motion feature matrices with feature matching degrees greater than feature matching degree thresholds are selected. The first aircraft flight data corresponding to the first motion feature matrix with feature matching degree greater than feature matching degree threshold and the second aircraft flight data corresponding to the second motion feature matrix with feature matching degree greater than feature matching degree threshold are determined to be aircraft flight data belonging to the same aircraft. The first aircraft flight data and the second aircraft flight data belonging to the same aircraft are combined to obtain the set of aircraft flight data belonging to the same aircraft.

[0134] In one example, the first motion feature matrix of the current aircraft is merged with each second motion feature matrix in the current airspace to obtain a joint feature vector set of the current aircraft. The current joint feature vector is input into a pre-trained random forest model to obtain the feature matching degree between the first motion feature matrix and the current second motion feature matrix of the current aircraft. The feature matching degree between the first motion feature matrix of the current aircraft and each second motion feature matrix in the current airspace is obtained one by one. Second motion feature matrices with a feature matching degree greater than the feature matching degree threshold are selected to determine that the second aircraft flight data corresponding to the second motion feature matrix belongs to the current aircraft. The first aircraft flight data and the second aircraft flight data of the current aircraft are combined to obtain the aircraft flight data set of the current aircraft. The aircraft flight data set of the current aircraft includes: first aircraft identifier, first aircraft time data, first aircraft positioning data, second aircraft time data, second aircraft positioning data, second aircraft speed data, and second aircraft angle data.

[0135] In this embodiment, since the first aircraft flight data is collected by the aircraft itself, a pre-trained random forest model is used to select a second motion feature matrix whose feature matching degree with the first motion matrix of the current aircraft flight data is greater than the feature matching degree threshold. This determines the second aircraft flight data belonging to the current aircraft, and the two sets of aircraft flight data belonging to the current aircraft are combined to obtain a set of aircraft flight data belonging to the current aircraft. This facilitates subsequent use of aircraft flight data belonging to the same aircraft for flight trajectory fitting, further ensuring the accuracy of flight trajectory fitting and reducing the amount of data computation for flight trajectory fitting.

[0136] S205. Determine the flight trajectory of the aircraft based on the flight data set of aircraft belonging to the same aircraft.

[0137] In one possible implementation, before determining the flight trajectory of an aircraft based on a set of aircraft flight data belonging to the same aircraft, the method further includes:

[0138] The isolated forest algorithm is used to calculate the mean and standard deviation of flight data of the same aircraft from different sources.

[0139] A threshold range is constructed based on the mean and standard deviation. Flight data of aircraft that do not fall within the threshold range are identified as outliers. Outlier cleaning is performed on flight data of aircraft belonging to the same aircraft to obtain cleaned flight data. The flight trajectory of the aircraft is then determined based on the cleaned flight data.

[0140] In one example, the first aircraft identifier of the current aircraft is determined as the aircraft identifier of the current aircraft. Using the Isolation Forest algorithm, the mean and standard deviation of the first aircraft positioning data of the current aircraft and the positioning data of all second aircraft belonging to the same aircraft are calculated. Based on the Laida criterion, a threshold range is constructed using the mean and standard deviation. It is then determined whether the positioning data of the first aircraft and all second aircraft belonging to the same aircraft are within the threshold range. Aircraft positioning data outside the threshold range are identified as outliers. These outliers are filtered out to obtain cleaned aircraft positioning data.

[0141] Using the Isolation Forest algorithm, the mean and standard deviation of the first aircraft time data of the current aircraft and the second aircraft time data belonging to the same aircraft are calculated. Based on the Laida criterion, a threshold range is constructed using the mean and standard deviation. It is then determined whether the first aircraft time data of the current aircraft and the second aircraft time data belonging to the same aircraft are within the threshold range. Aircraft time data outside the threshold range are identified as outliers. These outliers are then filtered out to obtain cleaned aircraft time data.

[0142] In this embodiment, the Isolation Forest algorithm has the characteristics of linear time complexity and low memory overhead. By removing aircraft flight data that are obviously outliers, the algorithm removes low-quality or erroneous aircraft flight data, improves the accuracy of aircraft flight data used to fit aircraft flight trajectories, and thus improves the aircraft flight trajectory fitting effect.

[0143] In one possible implementation, determining the flight trajectory of an aircraft based on a set of aircraft flight data belonging to the same aircraft includes:

[0144] Set hash indices for the spatiotemporal cubes corresponding to the first and second aircraft flight data belonging to the same aircraft, respectively. Based on the hash indices, deduplicate the first and second aircraft flight data belonging to the same aircraft to obtain deduplicated aircraft flight data at multiple time points.

[0145] The Kalman filter algorithm is used to filter the deduplicated flight data of the aircraft at each time point to determine the flight data of the target aircraft.

[0146] When there is a lack of target aircraft flight data at the current time point, cubic spline interpolation is used to fill in the target aircraft flight data at the current time point until target aircraft flight data exists at each time point. The target aircraft flight data at each time point are combined to obtain fused aircraft flight data. Based on the fused aircraft flight data, the flight trajectory of the aircraft is determined.

[0147] In one example, the current airspace is divided into several three-dimensional grids based on longitude, latitude, and altitude. The resolution of the three-dimensional grids is set according to accuracy requirements and is not limited in this application. The time axis is divided into fixed windows. A unique hash index is generated for each set of first and second aircraft flight data, where the hash index is generated based on the longitude, latitude, altitude, and time of the aircraft flight data. Based on the hash indices corresponding to the first and second aircraft flight data, it is determined whether the first and second aircraft flight data are within the same spatiotemporal cube. If the first and second aircraft flight data are within the same spatiotemporal cube, the aircraft flight data within the same spatiotemporal cube is determined to be spatiotemporally duplicated. The aircraft flight data with the highest confidence or the highest data source priority is selected, and other redundant points are deleted to obtain deduplicated aircraft flight data at multiple time points. The method for determining confidence and data source priority is not limited in this application.

[0148] The Kalman filter algorithm is used to estimate and predict the real-time operating status of the deduplicated aircraft flight data at each time point, so as to obtain the predicted flight data corresponding to each time point. Based on the estimation difference between the deduplicated aircraft flight data at the current time point and the corresponding predicted flight data, the target aircraft flight data at the current time point is determined.

[0149] When target aircraft flight data is lacking at the current time point, a cubic spline function is constructed based on the aircraft flight data from adjacent time points. Using this cubic spline function, interpolation is performed at time points lacking target aircraft flight data until target aircraft flight data exists at each time point. The interpolated target aircraft flight data from each time point are then combined to obtain fused aircraft flight data. The fused aircraft flight data is then used to fit the aircraft's flight trajectory.

[0150] In this embodiment, redundant trajectory points caused by overlapping observations from multiple acquisition methods are eliminated by setting a spatiotemporal cube and its corresponding hash index; asynchronous / heterogeneous aircraft flight data acquired by different acquisition methods are fused using the Kalman filter algorithm to achieve optimal estimation of the aircraft's dynamic state; when target aircraft flight data is lacking at the current time point, cubic spline interpolation is used to generate a smooth trajectory that conforms to kinematic laws, thereby generating the aircraft's flight trajectory and further improving the aircraft's flight trajectory fitting effect.

[0151] The aircraft trajectory fitting method based on multimodal information provided in this application acquires flight data of different modes of aircraft collected through various acquisition methods. This multimodal flight data can more comprehensively and accurately characterize the flight of the aircraft, facilitating subsequent trajectory fitting. By filtering out flight data that does not belong to the aircraft, the required flight data for trajectory fitting is reduced, further improving the speed and accuracy of trajectory fitting. Spatiotemporal alignment of flight data from different sources standardizes the data of different modes to the same spatiotemporal context, avoiding inaccuracies in trajectory fitting due to inconsistencies between different modes. A motion feature matrix is ​​used to distinguish flight data sets belonging to the same aircraft from those of different aircraft, facilitating subsequent trajectory fitting using data belonging to the same aircraft. Finally, the flight data of the same aircraft is fused to determine the aircraft's trajectory. The difference between the fitted UAV flight trajectory and the actual flight trajectory of the UAV is small, achieved by merging flight data from the same aircraft and the data from different modes acquired through various acquisition methods, thus improving the UAV trajectory fitting effect and consequently enhancing UAV monitoring.

[0152] Figure 3A schematic diagram of the structure of the aircraft trajectory fitting device based on multimodal information provided in this application is shown below. Figure 3 As shown, the aircraft trajectory fitting device based on multimodal information provided in this embodiment includes:

[0153] The data acquisition module 301 is used to acquire flight data of flying objects from different sources in the current airspace. Different sources are used to represent different acquisition methods, and different acquisition methods correspond to different modes. Among them, the flying objects include at least one moving object in the air.

[0154] The data filtering module 302 is used to filter flight data, filtering out flight data that does not belong to the aircraft, and obtaining the aircraft flight data;

[0155] The feature matrix construction module 303 is used to construct motion feature matrices corresponding to different sources of aircraft flight data, and to align the aircraft flight data from different sources in time and space to obtain aligned aircraft flight data.

[0156] The feature matching degree evaluation module 304 is used to evaluate the feature matching degree of the aligned aircraft flight data based on the motion feature matrix to obtain a set of aircraft flight data belonging to the same aircraft.

[0157] The trajectory determination module 305 is used to determine the flight trajectory of an aircraft based on a set of aircraft flight data belonging to the same aircraft.

[0158] In one possible implementation, the flight data includes data of the first aircraft and flight data of all aircraft within the current airspace; the data acquisition module 301 is specifically used for:

[0159] The first aircraft data is collected by sensors installed on the aircraft. The first aircraft data includes: first aircraft positioning data, first aircraft identification and first aircraft time data.

[0160] By setting up an airspace management center on the ground, flight data of all flying objects in the current airspace is collected. The flight data includes: flight time data, flight position data, flight speed data, and flight angle data. The airspace management center includes at least: several 5G-A base stations, an ADS-B system, an integrated ground surveillance base station, and different types of sensors and radar systems. The integrated ground surveillance base station is used to represent ground base stations with aircraft surveillance capabilities.

[0161] In one possible implementation, the data filtering module 302 is specifically used for:

[0162] Divide the current airspace into several sub-airspaces and generate a list of geolocation codes for each sub-airspace.

[0163] Insert the positioning data of each flying object into the spatial index, and combine it with the geolocation code list to obtain the trajectory bounding box corresponding to the positioning data of the flying object;

[0164] Calculate the correlation between different trajectory bounding boxes;

[0165] Based on correlation, flight data of flight bodies that do not belong to aircraft were identified;

[0166] The flight data of the aircraft that does not belong to the aircraft is filtered to obtain the second aircraft data. The second aircraft data and the first aircraft data are combined to obtain the aircraft flight data. The second aircraft data includes: second aircraft time data, second aircraft positioning data, second aircraft speed data and second aircraft angle data.

[0167] In one possible implementation, the motion characteristic matrix includes average velocity, velocity variation coefficient, and acceleration spectrum;

[0168] The feature matrix construction module 303 is specifically used for:

[0169] Based on the positioning data and time data of the first aircraft, the first average velocity, the first velocity variation coefficient and the first acceleration spectrum are calculated, and the first motion feature matrix is ​​constructed.

[0170] Based on the second aircraft's time data, second aircraft's positioning data, second aircraft's velocity data, and second aircraft's angle data, the second average velocity, the second velocity variation coefficient, and the second acceleration spectrum are calculated, and the second motion feature matrix is ​​constructed.

[0171] In one possible implementation, the feature matrix construction module 303 is further specifically used for:

[0172] Time alignment is performed on the time data of the first and second aircraft to obtain time-aligned data of the first and second aircraft.

[0173] Calculate the cosine similarity between the first motion feature matrix and the second motion feature matrix. Based on the cosine similarity, construct a first spatial rotation matrix. Using the first spatial rotation matrix, spatially align the first and second aircraft flight data to obtain aligned first and second aircraft data. Use the aligned first and second aircraft data as the aligned aircraft data.

[0174] In one possible implementation, the feature matching degree evaluation module 304 is specifically used for:

[0175] Input the first motion feature matrix and the second motion feature matrix into a pre-trained random forest model, and output the feature matching degree between the first motion feature matrix and the second motion feature matrix;

[0176] First motion feature matrices and second motion feature matrices with feature matching degrees greater than feature matching degree thresholds are selected. The first aircraft flight data corresponding to the first motion feature matrix with feature matching degree greater than feature matching degree threshold and the second aircraft flight data corresponding to the second motion feature matrix with feature matching degree greater than feature matching degree threshold are determined to be aircraft flight data belonging to the same aircraft. The first aircraft flight data and the second aircraft flight data belonging to the same aircraft are combined to obtain the set of aircraft flight data belonging to the same aircraft.

[0177] In one possible implementation, the trajectory determination module 305 is specifically used for:

[0178] Set hash indices for the spatiotemporal cubes corresponding to the first and second aircraft flight data belonging to the same aircraft, respectively. Based on the hash indices, deduplicate the first and second aircraft flight data belonging to the same aircraft to obtain deduplicated aircraft flight data at multiple time points.

[0179] The Kalman filter algorithm is used to filter the deduplicated flight data of the aircraft at each time point to determine the flight data of the target aircraft.

[0180] When there is a lack of target aircraft flight data at the current time point, cubic spline interpolation is used to fill in the target aircraft flight data at the current time point until target aircraft flight data exists at each time point. The target aircraft flight data at each time point are combined to obtain fused aircraft flight data. Based on the fused aircraft flight data, the flight trajectory of the aircraft is determined.

[0181] In one possible implementation, the aircraft trajectory fitting device based on multimodal information is also specifically used for:

[0182] The isolated forest algorithm is used to calculate the mean and standard deviation of flight data of the same aircraft from different sources.

[0183] A threshold range is constructed based on the mean and standard deviation. Flight data of aircraft that do not fall within the threshold range are identified as outliers. Outlier cleaning is performed on flight data of aircraft belonging to the same aircraft to obtain cleaned flight data. The flight trajectory of the aircraft is then determined based on the cleaned flight data.

[0184] The aircraft trajectory fitting device based on multimodal information provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0185] Figure 4 A schematic diagram of the structure of the electronic device provided in this application. Figure 4 As shown, the electronic device 40 provided in this embodiment includes at least one processor 401 and a memory 402. Optionally, the device 40 further includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.

[0186] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0187] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0188] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0189] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0190] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0191] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0192] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method.

[0193] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0194] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0195] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0196] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0197] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0198] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0199] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0200] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A method for fitting aircraft trajectory based on multimodal information, characterized in that, include: The system acquires flight data of flying objects from different sources within the current airspace, where the different sources represent different acquisition methods, and different acquisition methods correspond to different modes; wherein, the flying objects include at least one moving object in the air. The flight data is filtered to remove data that does not belong to the aircraft, thus obtaining the aircraft flight data; Based on aircraft flight data from different sources, a motion feature matrix corresponding to each source is constructed, and the aircraft flight data from different sources are aligned in time and space to obtain aligned aircraft flight data; the motion feature matrix includes average velocity, velocity variation coefficient, and acceleration spectrum; Based on the motion feature matrix, the aligned aircraft flight data are evaluated for feature matching degree to obtain a set of aircraft flight data belonging to the same aircraft. This includes: merging the first motion feature matrix of the current aircraft with each second motion feature matrix in the current airspace to obtain a joint feature vector set of the current aircraft; inputting the current joint feature vector into a pre-trained random forest model to obtain the feature matching degree between the first motion feature matrix and the current second motion feature matrix of the current aircraft; obtaining the feature matching degree between the first motion feature matrix of the current aircraft and each second motion feature matrix in the current airspace one by one; filtering the second motion feature matrices whose feature matching degree with the first motion feature matrix is ​​greater than the feature matching degree threshold, determining that the second aircraft flight data corresponding to the second motion feature matrix belongs to the current aircraft; and combining the first aircraft flight data and the second aircraft flight data of the current aircraft to obtain the set of aircraft flight data of the current aircraft. Determining the flight trajectory of an aircraft based on the set of flight data belonging to the same aircraft includes: setting hash indices for the spatiotemporal cubes corresponding to the first and second aircraft flight data belonging to the same aircraft, wherein the hash indices are generated based on the longitude, latitude, altitude, and time of the aircraft flight data; dividing the current airspace into several three-dimensional grids according to longitude, latitude, and altitude; and dividing the time axis into fixed windows; based on the hash indices corresponding to the first and second aircraft flight data, determining whether the first and second aircraft flight data are within the same spatiotemporal cube; and determining whether the first and second aircraft flight data are within the same spatiotemporal cube. Within the same spatiotemporal cube, the flight data of the aircraft within the same spatiotemporal cube is determined to be spatiotemporally repetitive. The flight data of the aircraft with the highest confidence or the highest data source priority is selected to obtain deduplicated flight data at multiple time points. The Kalman filter algorithm is used to filter the deduplicated flight data at each time point to determine the target aircraft flight data. When the target aircraft flight data is missing at the current time point, cubic spline interpolation is used to fill the target aircraft flight data at the current time point until the target aircraft flight data exists at every time point. The target aircraft flight data at each time point are combined to obtain fused flight data. The flight trajectory of the aircraft is determined based on the fused flight data.

2. The method according to claim 1, characterized in that, The flight data includes data from the first aircraft and flight data from all aircraft within the current airspace; acquiring flight data from different sources within the current airspace includes: The first aircraft data is collected by sensors installed on the aircraft. The first aircraft data includes: first aircraft positioning data, first aircraft identification and first aircraft time data. By setting up an airspace management center on the ground, flight data of all flying objects in the current airspace is collected. The flight data includes flight time data, flight position data, flight speed data, and flight angle data. The airspace management center includes at least several 5G-A base stations, an ADS-B system, an integrated ground surveillance base station, sensors, and radar systems. The integrated ground surveillance base station is used to represent ground base stations with aircraft surveillance capabilities.

3. The method according to claim 2, characterized in that, The process of filtering the flight data to obtain flight data that does not belong to the aircraft includes: Divide the current airspace into several sub-airspaces and generate a list of geographic location codes corresponding to each sub-airspace. Insert each of the aforementioned flight object positioning data into a spatial index, and combine it with the geographic location coding list to obtain the trajectory bounding box corresponding to the flight object positioning data; Calculate the correlation between the different bounding boxes of the stated trajectories; Based on the aforementioned correlation, flight data of flight bodies that do not belong to aircraft are determined; The flight data of the non-aircraft body is filtered to obtain the second aircraft data. The second aircraft data and the first aircraft data are combined to obtain the aircraft flight data. The second aircraft data includes: second aircraft time data, second aircraft positioning data, second aircraft speed data and second aircraft angle data.

4. The method according to claim 3, characterized in that, The construction of motion feature matrices based on aircraft flight data from different sources includes: Based on the first aircraft positioning data and the first aircraft time data, the first average velocity, the first velocity variation coefficient and the first acceleration spectrum are calculated, and the first motion feature matrix is ​​constructed. Based on the second aircraft's time data, second aircraft's positioning data, second aircraft's velocity data, and second aircraft's angle data, the second average velocity, the second velocity variation coefficient, and the second acceleration spectrum are calculated, and the second motion feature matrix is ​​constructed.

5. The method according to claim 4, characterized in that, The process of aligning aircraft flight data from different sources in time and space to obtain aligned aircraft flight data includes: The time data of the first aircraft and the time data of the second aircraft are time-aligned to obtain time-aligned first aircraft data and time-aligned second aircraft data; Calculate the cosine similarity between the first motion feature matrix and the second motion feature matrix. Based on the cosine similarity, construct a first spatial rotation matrix. Using the first spatial rotation matrix, spatially align the first aircraft flight data and the second aircraft flight data to obtain aligned first aircraft data and aligned second aircraft data. Use the aligned first aircraft data and aligned second aircraft data as aligned aircraft data.

6. The method according to claim 5, characterized in that, The step involves evaluating the feature matching degree of the aligned aircraft flight data based on the motion feature matrix to obtain a set of aircraft flight data belonging to the same aircraft, including: Input the first motion feature matrix and the second motion feature matrix into a pre-trained random forest model, and output the feature matching degree between the first motion feature matrix and the second motion feature matrix; The first motion feature matrix and the second motion feature matrix with a feature matching degree greater than the feature matching degree threshold are filtered to determine that the first aircraft flight data corresponding to the first motion feature matrix with a feature matching degree greater than the feature matching degree threshold and the second aircraft flight data corresponding to the second motion feature matrix belong to the same aircraft flight data. The first aircraft flight data and the second aircraft flight data belonging to the same aircraft are combined to obtain the set of aircraft flight data belonging to the same aircraft.

7. The method according to claim 1, characterized in that, Before determining the flight trajectory of an aircraft based on the set of aircraft flight data belonging to the same aircraft, the method further includes: The Isolation Forest algorithm was used to calculate the mean and standard deviation of flight data from different sources belonging to the same aircraft. A threshold range is constructed based on the mean and standard deviation. Flight data of aircraft that do not belong to the threshold range are identified as outliers. Outlier cleaning is performed on the flight data of aircraft belonging to the same aircraft to obtain cleaned flight data. The flight trajectory of the aircraft is then determined based on the cleaned flight data.

8. A device for fitting aircraft trajectory based on multimodal information, characterized in that, include: The data acquisition module is used to acquire flight data of flying objects from different sources within the current airspace. The different sources are used to represent different acquisition methods, and different acquisition methods correspond to different modes. The flying objects include at least one moving object in the air. The data filtering module is used to filter the flight data, filtering out flight data that does not belong to the aircraft, and obtaining the aircraft flight data; The feature matrix construction module is used to construct motion feature matrices corresponding to different sources of aircraft flight data, and to align the aircraft flight data from different sources in time and space to obtain aligned aircraft flight data; the motion feature matrix includes average velocity, velocity variation coefficient, and acceleration spectrum; The feature matching degree evaluation module is used to evaluate the feature matching degree of the aligned aircraft flight data based on the motion feature matrix to obtain a set of aircraft flight data belonging to the same aircraft. This includes: merging the first motion feature matrix of the current aircraft with each second motion feature matrix in the current airspace to obtain a joint feature vector set of the current aircraft; inputting the current joint feature vectors into a pre-trained random forest model to obtain the feature matching degree between the first motion feature matrix and the current second motion feature matrix of the current aircraft; obtaining the feature matching degree between the first motion feature matrix of the current aircraft and each second motion feature matrix in the current airspace one by one; filtering second motion feature matrices whose feature matching degree with the first motion feature matrix is ​​greater than a feature matching degree threshold, determining that the second aircraft flight data corresponding to the second motion feature matrix belongs to the current aircraft; and combining the first and second aircraft flight data of the current aircraft to obtain the set of aircraft flight data of the current aircraft. The trajectory determination module is used to determine the flight trajectory of an aircraft based on the set of aircraft flight data belonging to the same aircraft. This includes: setting hash indices for the spatiotemporal cubes corresponding to the first and second aircraft flight data belonging to the same aircraft, respectively. The hash indices are generated based on the longitude, latitude, altitude, and time of the aircraft flight data; the current airspace is divided into several three-dimensional grids according to longitude, latitude, and altitude; and the time axis is divided into fixed windows. Based on the hash indices corresponding to the first and second aircraft flight data, it is determined whether the first and second aircraft flight data are within the same spatiotemporal cube. When the flight data of an aircraft is within the same spatiotemporal cube, it is determined that the flight data within the same spatiotemporal cube is spatiotemporally repeated. The flight data with the highest confidence or the highest data source priority is selected to obtain deduplicated flight data at multiple time points. The Kalman filter algorithm is used to filter the deduplicated flight data at each time point to determine the target aircraft flight data. When the target aircraft flight data is missing at the current time point, cubic spline interpolation is used to fill the target aircraft flight data at the current time point until the target aircraft flight data exists at every time point. The target aircraft flight data at each time point are combined to obtain fused flight data. The flight trajectory of the aircraft is determined based on the fused flight data.

9. An electronic device, characterized in that, include: Memory, processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-7.

11. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images