Aircraft trajectory fitting method and device based on multi-modal information and storage medium

Abstract

The embodiment of the invention provides an aircraft trajectory fitting method and device based on multi-modal information and a storage medium. The method comprises the following steps: acquiring flight data of flight bodies from different sources in a current airspace; screening the flight data, and filtering the flight data not belonging to the aircraft; based on the flight data of the aircrafts from different sources, constructing motion feature matrixes corresponding to the sources, and aligning the flight data of the aircrafts from different sources in time and space; according to the motion feature matrix, performing feature matching degree evaluation on the aligned aircraft flight data to obtain an aircraft flight data set belonging to the same aircraft; and determining the flight path of the aircraft according to the aircraft flight data set belonging to the same aircraft. According to the method, the difference between the flight path of the unmanned aerial vehicle obtained through fitting and the real flight path of the unmanned aerial vehicle is small, the flight path fitting effect of the unmanned aerial vehicle is improved, and then the monitoring effect of the unmanned aerial vehicle can be improved.

Description

Technical Field

[0001] The present application relates to data processing technology, and in particular to a method, device and storage medium for fitting aircraft trajectories based on multimodal information. Background Art

[0002] In recent years, drone technology has made significant progress, with continuous optimization in flight control, navigation and positioning, payload loading, and communication transmission. Deep integration with technologies like 5G and artificial intelligence has further expanded its application scenarios and market potential. Drones have a wide range of applications and a vast market potential, and their industry applications are becoming increasingly diverse and in-depth, while the development of related supporting systems is also becoming increasingly sophisticated.

[0003] In the related art, real-time monitoring of drones requires fitting the drone's flight trajectory. This fitting is then used to monitor the drone in real time. However, existing drone flight data is collected using only sensors or radar. This leads to significant discrepancies between the fitted drone flight trajectory and the actual flight trajectory, resulting in poor tracking and monitoring.

[0004] Therefore, in order to improve the UAV flight trajectory fitting effect, it is necessary to provide a UAV trajectory fitting method based on multimodal information. Summary of the Invention

[0005] The embodiments of the present application provide a method, device, and storage medium for aircraft trajectory fitting based on multimodal information to solve the problem of poor UAV flight trajectory fitting effect.

[0006] In a first aspect, an embodiment of the present application provides an aircraft trajectory fitting method based on multimodal information, comprising:

[0007] Acquire flight data of flying objects from different sources within the current airspace, where different sources are used to represent different acquisition methods, and different acquisition methods correspond to different modalities; wherein the flying object includes at least one moving object in the air;

[0008] Filter the flight data, filter out the flight data that does not belong to the aircraft, and obtain the aircraft flight data;

[0009] Based on the aircraft flight data from different sources, a motion feature matrix of the corresponding source is constructed, and the aircraft flight data from different sources are aligned in time and space to obtain the aligned aircraft flight data;

[0010] According to the motion feature matrix, the feature matching degree of the aligned aircraft flight data is evaluated to obtain the aircraft flight data set belonging to the same aircraft;

[0011] The flight trajectory of the aircraft is determined based on a set of aircraft flight data belonging to the same aircraft.

[0012] In one possible implementation, the flight data includes the first aircraft data and flight data of all aircraft in the current airspace; obtaining the flight data of aircraft from different sources in the current airspace includes:

[0013] The first aircraft data of the aircraft itself is collected by a sensor provided on the aircraft, the first aircraft data including: first aircraft positioning data, first aircraft identification and first aircraft time data;

[0014] The flight data of all flying objects in the current airspace are collected through the airspace management center set up on the ground, where the flight data include: flight time data, flight positioning data, flight speed data and flight angle data; the airspace management center includes at least: several 5G-A base stations, ADS-B systems, integrated fusion ground surveillance base stations, different types of sensors and radar systems, and the integrated fusion ground surveillance base stations are used to represent ground base stations with aircraft monitoring functions.

[0015] In one possible implementation, screening the flight data to filter out flight data that does not belong to the aircraft to obtain the aircraft flight data includes:

[0016] Divide the current airspace into several sub-airspaces and generate a geographic location code list corresponding to each sub-airspace;

[0017] Insert each flying object's positioning data into the spatial index and combine it with the geographic location code list to obtain the trajectory bounding box corresponding to the flying object's positioning data;

[0018] Calculate the correlation between different trajectory bounding boxes;

[0019] Determine the flight data of the flying object that does not belong to the aircraft based on the correlation;

[0020] The flight data of the flying object that does not belong to the aircraft is filtered to obtain second aircraft data, and the second aircraft data is combined with the first aircraft data to obtain aircraft flight data, wherein the second aircraft data includes: second aircraft time data, second aircraft positioning data, second aircraft speed data, and second aircraft angle data.

[0021] In one possible implementation, the motion feature matrix includes average velocity, velocity variation coefficient, and acceleration spectrum;

[0022] Based on the flight data of aircraft from different sources, the motion feature matrix of the corresponding source is constructed, including:

[0023] Based on the first aircraft positioning data and the first aircraft time data, a first average velocity, a first velocity variation coefficient and a first acceleration spectrum are calculated to construct a first motion feature matrix;

[0024] Based on the second aircraft time data, the second aircraft positioning data, the second aircraft speed data and the second aircraft angle data, a second average speed, a second speed variation coefficient and a second acceleration spectrum are calculated to construct a second motion feature matrix.

[0025] In one possible implementation, aligning aircraft flight data from different sources in time and space to obtain aligned aircraft flight data includes:

[0026] Time-aligning the first aircraft time data and the second aircraft time data to obtain time-aligned first aircraft data and time-aligned second aircraft data;

[0027] Calculate the cosine similarity between the first motion feature matrix and the second motion feature matrix, construct a first spatial rotation matrix based on the cosine similarity, and spatially align the first aircraft flight data and the second aircraft flight data through the first spatial rotation matrix to obtain aligned first aircraft data and aligned second aircraft data, and use the aligned first aircraft data and the aligned second aircraft data as aligned aircraft data.

[0028] In one possible implementation, feature matching evaluation is performed on the aligned aircraft flight data according to the motion feature matrix to obtain an aircraft flight data set belonging to the same aircraft, including:

[0029] Inputting the first motion feature matrix and the second motion feature matrix into a pre-trained random forest model, and outputting a feature matching degree between the first motion feature matrix and the second motion feature matrix;

[0030] The first motion feature matrix and the second motion feature matrix having a feature matching degree greater than a feature matching degree threshold are screened, and it is determined that the first aircraft flight data corresponding to the first motion feature matrix and the second aircraft flight data corresponding to the second motion feature matrix having a feature matching degree greater than the feature matching degree threshold belong to aircraft flight data of the same aircraft, and the first aircraft flight data and the second aircraft flight data belonging to the same aircraft are combined to obtain an aircraft flight data set belonging to the same aircraft.

[0031] In one possible implementation, determining the flight trajectory of an aircraft based on a set of aircraft flight data belonging to the same aircraft includes:

[0032] Setting hash indexes of space-time cubes corresponding to first aircraft flight data and second aircraft flight data belonging to the same aircraft, respectively, and deduplicating the first aircraft flight data and the second aircraft flight data belonging to the same aircraft based on the hash indexes to obtain deduplicated aircraft flight data at multiple time points;

[0033] The Kalman filter algorithm is used to filter the deduplicated aircraft flight data at each time point to determine the target aircraft flight data;

[0034] When there is a lack of target aircraft flight data at the current time point, cubic spline interpolation is used to fill in the target aircraft flight data at the current time point until target aircraft flight data exists at each time point. The target aircraft flight data at each time point are combined to obtain fused aircraft flight data. The flight trajectory of the aircraft is determined based on the fused aircraft flight data.

[0035] In one possible implementation, before determining the flight trajectory of the aircraft based on the aircraft flight data set belonging to the same aircraft, the method further includes:

[0036] The mean and standard deviation of aircraft flight data belonging to the same aircraft from different sources are calculated using the isolation forest algorithm;

[0037] A threshold range is constructed based on the mean value and standard deviation, and aircraft flight data that does not fall within the threshold range is determined as an outlier. The aircraft flight data belonging to the same aircraft are cleaned of outliers to obtain the cleaned aircraft flight data, so as to determine the flight trajectory of the aircraft based on the cleaned aircraft flight data.

[0038] In a second aspect, an embodiment of the present application provides an aircraft trajectory fitting device based on multimodal information, comprising:

[0039] A data acquisition module is used to acquire flight data of flying objects from different sources in the current airspace, where different sources are used to represent different acquisition methods, and different acquisition methods correspond to different modalities; wherein the flying object includes at least one moving object in the air;

[0040] A data screening module is used to screen the flight data, filter out the flight data that does not belong to the aircraft, and obtain the aircraft flight data;

[0041] A feature matrix construction module is used to construct motion feature matrices of the corresponding sources based on aircraft flight data from different sources, and to align the aircraft flight data from different sources in time and space to obtain aligned aircraft flight data;

[0042] A feature matching evaluation module is used to evaluate the feature matching of the aligned aircraft flight data based on the motion feature matrix to obtain a set of aircraft flight data belonging to the same aircraft;

[0043] The trajectory determination module is used to determine the flight trajectory of the aircraft based on the aircraft flight data set belonging to the same aircraft.

[0044] In a possible implementation, the flight data includes the first aircraft data and the flight body data of all flying bodies in the current airspace; the data acquisition module is specifically configured to:

[0045] The first aircraft data of the aircraft itself is collected by a sensor provided on the aircraft, the first aircraft data including: first aircraft positioning data, first aircraft identification and first aircraft time data;

[0046] The flight data of all flying objects in the current airspace are collected through the airspace management center set up on the ground, where the flight data include: flight time data, flight positioning data, flight speed data and flight angle data; the airspace management center includes at least: several 5G-A base stations, ADS-B systems, integrated fusion ground surveillance base stations, different types of sensors and radar systems, and the integrated fusion ground surveillance base stations are used to represent ground base stations with aircraft monitoring functions.

[0047] In one possible implementation, the data screening module is specifically configured to:

[0048] Divide the current airspace into several sub-airspaces and generate a geographic location code list corresponding to each sub-airspace;

[0049] Insert each flying object's positioning data into the spatial index and combine it with the geographic location code list to obtain the trajectory bounding box corresponding to the flying object's positioning data;

[0050] Calculate the correlation between different trajectory bounding boxes;

[0051] Determine the flight data of the flying object that does not belong to the aircraft based on the correlation;

[0052] The flight data of the flying object that does not belong to the aircraft is filtered to obtain second aircraft data, and the second aircraft data is combined with the first aircraft data to obtain aircraft flight data, wherein the second aircraft data includes: second aircraft time data, second aircraft positioning data, second aircraft velocity data, and second aircraft angle data. In one possible embodiment, the motion feature matrix includes average velocity, velocity variation coefficient, and acceleration spectrum;

[0053] The feature matrix building module is specifically used to:

[0054] Based on the first aircraft positioning data and the first aircraft time data, a first average velocity, a first velocity variation coefficient and a first acceleration spectrum are calculated to construct a first motion feature matrix;

[0055] Based on the second aircraft time data, the second aircraft positioning data, the second aircraft speed data and the second aircraft angle data, a second average speed, a second speed variation coefficient and a second acceleration spectrum are calculated to construct a second motion feature matrix.

[0056] In a possible implementation, the feature matrix construction module is further specifically configured to:

[0057] Time-aligning the first aircraft time data and the second aircraft time data to obtain time-aligned first aircraft data and time-aligned second aircraft data;

[0058] Calculate the cosine similarity between the first motion feature matrix and the second motion feature matrix, construct a first spatial rotation matrix based on the cosine similarity, and spatially align the first aircraft flight data and the second aircraft flight data through the first spatial rotation matrix to obtain aligned first aircraft data and aligned second aircraft data, and use the aligned first aircraft data and the aligned second aircraft data as aligned aircraft data.

[0059] In a possible implementation, the feature matching evaluation module is specifically configured to:

[0060] Inputting the first motion feature matrix and the second motion feature matrix into a pre-trained random forest model, and outputting a feature matching degree between the first motion feature matrix and the second motion feature matrix;

[0061] The first motion feature matrix and the second motion feature matrix having a feature matching degree greater than a feature matching degree threshold are screened, and it is determined that the first aircraft flight data corresponding to the first motion feature matrix and the second aircraft flight data corresponding to the second motion feature matrix having a feature matching degree greater than the feature matching degree threshold belong to aircraft flight data of the same aircraft, and the first aircraft flight data and the second aircraft flight data belonging to the same aircraft are combined to obtain an aircraft flight data set belonging to the same aircraft.

[0062] In a possible implementation, the trajectory determination module is specifically configured to:

[0063] Setting hash indexes of space-time cubes corresponding to first aircraft flight data and second aircraft flight data belonging to the same aircraft, respectively, and deduplicating the first aircraft flight data and the second aircraft flight data belonging to the same aircraft based on the hash indexes to obtain deduplicated aircraft flight data at multiple time points;

[0064] The Kalman filter algorithm is used to filter the deduplicated aircraft flight data at each time point to determine the target aircraft flight data;

[0065] When there is a lack of target aircraft flight data at the current time point, cubic spline interpolation is used to fill in the target aircraft flight data at the current time point until target aircraft flight data exists at each time point. The target aircraft flight data at each time point are combined to obtain fused aircraft flight data. The flight trajectory of the aircraft is determined based on the fused aircraft flight data.

[0066] In a possible implementation, the aircraft trajectory fitting device based on multimodal information is further specifically used for:

[0067] The mean and standard deviation of aircraft flight data belonging to the same aircraft from different sources are calculated using the isolation forest algorithm;

[0068] A threshold range is constructed based on the mean value and standard deviation, and aircraft flight data that does not fall within the threshold range is determined as an outlier. The aircraft flight data belonging to the same aircraft are cleaned of outliers to obtain the cleaned aircraft flight data, so as to determine the flight trajectory of the aircraft based on the cleaned aircraft flight data.

[0069] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor;

[0070] Memory stores computer-executable instructions;

[0071] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementations of the first aspect.

[0072] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect above and / or various possible implementation methods of the first aspect.

[0073] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementation methods of the first aspect.

[0074] The multimodal information-based aircraft trajectory fitting method, device, and storage medium provided by the embodiments of the present application obtain flight data of an aircraft of different modes collected by multiple acquisition methods. The flight data of the aircraft of different modes can more comprehensively and accurately characterize the flight of the aircraft, facilitating the subsequent use of the flight data to fit the flight trajectory. By screening the flight data, flight data that does not belong to the aircraft is filtered out, reducing the flight data required for fitting the flight trajectory, and further improving the speed and accuracy of flight trajectory fitting. By performing spatiotemporal alignment on flight data from different sources, flight data of different modes are standardized to the same spatiotemporal time, avoiding inaccurate flight data fitting caused by inconsistencies between flight data of different modes. Using a motion feature matrix, flight data of aircraft belonging to the same aircraft are aggregated to distinguish flight data of different aircraft, facilitating the subsequent use of flight data belonging to the same aircraft for flight trajectory fitting. The aircraft data of the same aircraft are fused to determine the flight trajectory of the aircraft. The difference between the fitted flight trajectory of the drone using flight data of different modes collected by multiple acquisition methods and the actual flight trajectory of the drone is small, thereby improving the drone flight trajectory fitting effect and thereby improving the monitoring effect of the drone. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0076] Figure 1 Schematic diagram of the scenario for fitting the flight trajectory of the drone provided in this application;

[0077] Figure 2 A schematic diagram of the flow of the aircraft trajectory fitting method based on multimodal information provided in this application;

[0078] Figure 3 A schematic diagram of the structure of the aircraft trajectory fitting device based on multimodal information provided by this application;

[0079] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application.

[0080] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0081] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0082] First, let’s explain the terms involved in this application:

[0083] 5G-A (5G-Advanced, the fifth generation of mobile communication technology evolution) is based on the evolution and enhancement of 5G network functions and coverage. 5G-A base stations are base stations that support 5G-A networks and have Integrated Sensing and Communication (ISAC) capabilities. 5G-A base stations can not only transmit data but also realize environmental perception (such as vehicle detection and drone tracking).

[0084] ADS-B (Automatic dependent surveillance – broadcast) refers to air-to-air traffic surveillance between aircraft capable of broadcasting position reports.

[0085] Integrated fusion ground surveillance base station: refers to an integrated fusion ground surveillance base station for manned / unmanned aircraft. This base station can capture communication data between the drone and the remote controller, receive and parse the drone Remote ID broadcast signals that comply with multinational standards, as well as the aircraft's ADS-B messages, thereby obtaining and updating key information such as the identity, geographic location, and flight status of manned and unmanned aircraft in real time.

[0086] Figure 1 The schematic diagram of the scene of the UAV flight trajectory fitting provided in this application is as follows: Figure 1 As shown, the specific application scenario of the present application is to monitor a drone. Monitoring the drone may include: collecting the flight data of the drone through the acquisition device 101, and sending the flight data to the computing device 102; the computing device 102 uses the flight data of the drone to fit the flight trajectory of the drone to obtain the drone trajectory, and monitors the drone in real time according to the fitted drone trajectory.

[0087] Combined with the above scenario, it can be seen that in the existing technology, the flight data of the drone is only collected through one of the sensors and radars. There is a large difference between the fitted drone flight trajectory and the actual flight trajectory of the drone. The drone flight trajectory fitting effect is poor, resulting in poor monitoring effect of the drone. Technical problem.

[0088] The present application provides a multimodal information-based aircraft trajectory fitting method. By acquiring flight data of an aircraft of different modes acquired by multiple acquisition methods, the flight data of the aircraft of different modes can more comprehensively and accurately characterize the flight of the aircraft, facilitating the subsequent use of the flight data to fit the flight trajectory. By screening the flight data, flight data that does not belong to the aircraft is filtered out, reducing the flight data required for fitting the flight trajectory, and further improving the speed and accuracy of flight trajectory fitting. By performing spatiotemporal alignment on flight data from different sources, flight data of different modes are standardized to the same spatiotemporal time, avoiding inaccurate flight data fitting caused by inconsistencies between flight data of different modes. Using a motion feature matrix, flight data of aircraft belonging to the same aircraft are aggregated to distinguish flight data of different aircraft, facilitating the subsequent use of flight data belonging to the same aircraft for flight trajectory fitting. The aircraft data of the same aircraft are fused to determine the flight trajectory of the aircraft. The difference between the fitted flight trajectory of the unmanned aerial vehicle (UAV) and the actual flight trajectory of the UAV obtained by using flight data of different modes acquired by multiple acquisition methods is small, thereby improving the UAV flight trajectory fitting effect and thereby improving the monitoring effect of the UAV.

[0089] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0090] Figure 2 The flowchart of the aircraft trajectory fitting method based on multimodal information provided in this application is as follows: Figure 2 As shown, the method includes:

[0091] S201. Acquire flight data of flying objects from different sources in the current airspace.

[0092] In a possible implementation, different sources are used to represent different acquisition methods, and different acquisition methods correspond to different modalities; the flying object includes at least one moving object in the air.

[0093] In some embodiments, the flight data includes the first aircraft data and the flight data of all flying objects in the current airspace; obtaining the flight data of flying objects from different sources in the current airspace includes:

[0094] The first aircraft data of the aircraft itself is collected by a sensor provided on the aircraft, the first aircraft data including: first aircraft positioning data, first aircraft identification and first aircraft time data;

[0095] The flight data of all flying objects in the current airspace are collected through the airspace management center set up on the ground, where the flight data include: flight time data, flight positioning data, flight speed data and flight angle data; the airspace management center includes at least: several 5G-A base stations, ADS-B systems, integrated fusion ground surveillance base stations, different types of sensors and radar systems, and the integrated fusion ground surveillance base stations are used to represent ground base stations with aircraft monitoring functions.

[0096] In one example, each airspace has a longitude range, a latitude range, and an altitude range. The first aircraft positioning data may include the longitude, latitude, and altitude of the aircraft. A flight control system provided in the aircraft obtains a first aircraft identifier, measures the longitude, latitude, and altitude of the aircraft, and obtains the first aircraft positioning data based on the longitude, latitude, and altitude of the aircraft. A determination is made as to whether the longitude, latitude, and altitude of the aircraft are within the longitude range, latitude range, and altitude range of the current airspace. If the longitude, latitude, and altitude of the aircraft are within the longitude range, latitude range, and altitude range of the current airspace, the aircraft is determined to belong to the current airspace. The flight control system provided in the aircraft timestamps the first aircraft positioning data to obtain first aircraft time data.

[0097] Optionally, the flying body data can be collected by the 5G-A base station: the 5G-A base station transmits a wireless signal for sensing, wherein the wireless signal is reflected, scattered or diffracted when encountering a flying body in the current airspace, and a feedback signal is obtained, and the feedback signal contains the spatial and motion information of the flying body; the 5G-A base station processes the feedback signal through a signal processing algorithm to obtain the flying body data, which may include: obtaining the flying body time data of the flying body through the timestamp of transmitting and receiving the feedback signal through the wireless signal; the 5G-A base station obtains the distance and arrival angle of the flying body based on the feedback signal, and calculates the position of the flying body in three-dimensional space in combination with the geographical location information of the 5G-A base station itself to obtain the flying body positioning data; the 5G-A base station obtains the longitudinal velocity of the flying body through Doppler frequency shift, and decomposes the radial velocity into a three-dimensional coordinate system by differentiating the continuous position changes of the flying body in three-dimensional space and combining the arrival angle to obtain the flying body flight velocity data; the 5G-A base station calculates the heading angle data through the direction of the flying body velocity vector to obtain the flying body angle data of the flying body.

[0098] Optionally, flight body data can be collected through the ADS-B system: the ADS-B system includes a global satellite navigation system and an inertial navigation system. The flight body time data is obtained through the high-precision UTC (coordinated universal time) provided by the global satellite navigation system; the three-dimensional coordinates of the flight body are solved in real time by combining the global satellite navigation system with the inertial navigation system to obtain the flight body positioning data; the flight body speed data is obtained through vector synthesis and dynamic pressure calculation using the ground speed and track direction angle provided by the global satellite navigation system; the track direction angle is corrected through the global satellite navigation system to obtain the flight body angle data.

[0099] Optionally, the flight object data can be collected through an integrated fusion ground surveillance base station: the integrated fusion ground surveillance base station has at least radio spectrum detection capability, Remote ID (remote identity identification) broadcast signal reception and analysis capability, and ADS-B message reception and analysis capability. Through the radio spectrum detection capability and Remote ID broadcast signal reception and analysis capability of the integrated fusion ground surveillance base station, the latitude and longitude, altitude, speed and other data of the flight object are obtained, and the flight object positioning data and flight object speed data are obtained; through the ADS-B message reception and analysis capability of the integrated fusion ground surveillance base station, the ADS-B message of the aircraft is received and analyzed to obtain the flight time data and flight angle data of the flight object.

[0100] Optionally, the flight body data can be collected through different types of sensors: the sensors include at least a global satellite navigation system and an inertial navigation system, and the satellite navigation system is used to perform atomic clock timing to obtain the flight body time data; the satellite navigation system is combined with a barometric altimeter to obtain the flight body's longitude, latitude, and geometric altitude data to obtain the flight body positioning data; the ground speed and track direction angle collected by the satellite navigation system are combined with the airspeed measured by the Pitot tube sensor to calculate the flight body speed data; the flight body's heading data is measured by the inertial navigation system, and combined with the airflow sensor for auxiliary correction to obtain the flight body angle data.

[0101] Optionally, the flying body data can be collected by a radar system: using a high-precision clock inside the radar (such as a rubidium atomic clock or a constant-temperature crystal oscillator, etc.) as a time reference, and measuring the time difference between the emission of the radar electromagnetic wave and the reception of the echo, the flying body time data is calculated; the flying body angle data is determined by mechanically scanning the radar by pointing the antenna at the strongest echo, or the beam pointing is controlled by the phase shifter of the phased array radar (electronic scanning) to output the flying body angle data in real time; the flying body is measured for distance and height by the radar system, and the flying body angle data is converted into a geodetic coordinate system to obtain the flying body positioning data; the radar system uses the frequency offset of the echo signal to calculate the radial velocity, and combined with differential velocity measurement, the flying body velocity data is obtained.

[0102] In this embodiment, the first aircraft data of the aircraft itself is collected by sensors installed on the aircraft; the flight body data of all flying objects in the airspace are collected by different types of airspace management centers installed on the ground. The flight data is more comprehensive and has more modalities, thereby improving the subsequent flight trajectory fitting effect of the flight data collected by various collection methods, further improving the data quality and the monitoring effect of the drone.

[0103] S202: Filter the flight data to filter out flight data that does not belong to the aircraft, and obtain aircraft flight data.

[0104] In one possible implementation, screening the flight data to filter out flight data that does not belong to the aircraft to obtain the aircraft flight data includes:

[0105] Divide the current airspace into several sub-airspaces and generate a geographic location code list corresponding to each sub-airspace;

[0106] Insert each flying object's positioning data into the spatial index and combine it with the geographic location code list to obtain the trajectory bounding box corresponding to the flying object's positioning data;

[0107] Calculate the correlation between different trajectory bounding boxes;

[0108] Determine the flight data of the flying object that does not belong to the aircraft based on the correlation;

[0109] The flight data of the flying object that does not belong to the aircraft is filtered to obtain second aircraft data, and the second aircraft data is combined with the first aircraft data to obtain aircraft flight data, wherein the second aircraft data includes: second aircraft time data, second aircraft positioning data, second aircraft speed data, and second aircraft angle data.

[0110] In one example, a Geohash (geocode system) grid system can be used to divide the current airspace into several sub-airspaces, encode the geographic location of each sub-airspace, and combine to generate a geographic location code list corresponding to each sub-airspace in the current airspace.

[0111] The current aircraft's location data for all times collected using the current acquisition method, from the initial detection time of the current aircraft in the current airspace to the current time, is inserted into the spatial index. Each leaf node in the spatial index stores the corresponding aircraft location data. The root node of the spatial index automatically calculates and stores the union of the minimum bounding rectangles (MBRs) of all leaf nodes below it. Specifically, starting from the root node of the spatial index, the spatial index is accessed layer by layer downwards: if the current node is a leaf node, the MBR of the current node's location data is calculated and stored. If the current node is not a leaf node, the child nodes of the current node are accessed until the current node is a leaf node. The MBR of the current leaf node's location data is calculated and stored. The MBRs of all leaf nodes are then taken as the union to obtain the trajectory bounding box corresponding to the trajectory of the current aircraft obtained using the current acquisition method. All sub-airspaces covered by the trajectory bounding box are calculated, and a set of geographic location codes corresponding to the coverage of the trajectory bounding box is generated. Calculate the trajectory bounding boxes and corresponding geographic location codes corresponding to the flight data collected by different acquisition methods for all flight objects in the current airspace, calculate the set similarity between the sets of geographic location codes corresponding to the trajectory bounding boxes, and determine the set similarity as the correlation between the trajectory bounding boxes corresponding to the sets of geographic location codes. The set similarity between Geohash prefix sets can be calculated using methods such as the Jaccard similarity coefficient, overlap coefficient, and Dice coefficient. The method for calculating set similarity is not limited in this application.

[0112] Since the motion characteristics of aircraft and non-aircraft flying objects are different, the correlation between the aircraft data of an aircraft and the aircraft data of other aircraft is higher than the correlation threshold, the correlation between the aircraft data of an aircraft and the flight data of non-aircraft flying objects is lower than the correlation threshold, and the correlation between the flight data of non-aircraft flying objects and the flight data of non-aircraft flying objects is lower than the correlation threshold.

[0113] Optionally, determining flight data of an aircraft that does not belong to an aircraft based on correlation can be achieved through the following steps: obtaining all correlation sets corresponding to each piece of aircraft data; when the number of correlations greater than a correlation threshold in the correlation set corresponding to the current aircraft's flight data is less than or equal to a quantity threshold, determining the current aircraft to be a non-aircraft, and filtering the current aircraft's flight data; and when the number of correlations greater than the correlation threshold in the correlation set corresponding to the current aircraft's flight data is greater than a quantity threshold, determining the current aircraft to be a suspected aircraft. The quantity threshold can be calculated as C (n,2) = n! / (2! (n-2)!), where n is the number of types of airspace management centers located on the ground.

[0114] According to a pre-constructed aircraft flight trajectory set, the features of the trajectory bounding box corresponding to the pre-constructed aircraft flight trajectory set are extracted, wherein the pre-constructed aircraft flight trajectory set conforms to the aircraft behavior model. The features are used to train a bounding box classifier to obtain a trained bounding box classifier. The trained bounding box classifier is used to classify and identify the trajectory bounding box of the current suspected aircraft, and identify whether the trajectory bounding box of the current suspected aircraft belongs to an aircraft, wherein the trained threshold classifier has the ability to classify and identify the flight data of the flying object. When the trajectory bounding box of the current suspected aircraft does not belong to an aircraft, the flight data of the trajectory bounding box of the current suspected aircraft is filtered; when the trajectory bounding box of the current suspected aircraft belongs to an aircraft, the current suspected aircraft is determined to be an aircraft, and the second aircraft data of the current aircraft is obtained, wherein the second aircraft data includes: second aircraft time data, second aircraft positioning data, second aircraft speed data and second aircraft angle data.

[0115] In this embodiment, spatial indexing is used for a priori bounding box collision detection, quickly eliminating flight data that does not belong to the aircraft. Only the flight volume data belonging to the aircraft is retained for subsequent drone trajectory fitting. This reduces the computational workload of the flight data and further improves the accuracy and effectiveness of trajectory fitting. Geolocation encoding further compresses spatial information and reduces computational complexity.

[0116] S203 . Based on the aircraft flight data from different sources, construct a motion feature matrix of the corresponding source, and align the aircraft flight data from different sources in time and space to obtain aligned aircraft flight data.

[0117] In one possible implementation, the motion feature matrix includes average velocity, velocity variation coefficient, and acceleration spectrum;

[0118] Based on the flight data of aircraft from different sources, the motion feature matrix of the corresponding source is constructed, including:

[0119] Based on the first aircraft positioning data and the first aircraft time data, a first average velocity, a first velocity variation coefficient and a first acceleration spectrum are calculated to construct a first motion feature matrix;

[0120] Based on the second aircraft time data, the second aircraft positioning data, the second aircraft speed data and the second aircraft angle data, a second average speed, a second speed variation coefficient and a second acceleration spectrum are calculated to construct a second motion feature matrix.

[0121] In one example, the first aircraft positioning data includes the first initial positioning data of the current aircraft entering the current airspace and the current first real-time positioning data of the aircraft. The first aircraft time data includes the first initial time data of the current aircraft entering the current airspace and the current first real-time time data of the aircraft. A first distance between the current first real-time positioning data of the aircraft and the first initial positioning data of the current aircraft entering the current airspace is calculated. A first total time between the current first real-time time data of the aircraft and the first initial time data of the current aircraft entering the current airspace is calculated. The first total distance is divided by the first total time to obtain a first average speed. A first speed standard deviation is calculated based on the first average speed. The ratio between the first speed standard deviation and the first average speed is calculated to obtain a first speed coefficient of variation. A first instantaneous speed sequence is calculated based on the first aircraft positioning data and the first aircraft time data. A first instantaneous acceleration sequence is calculated based on the first instantaneous speed sequence. The first instantaneous speed sequence and the first instantaneous acceleration sequence are combined to construct a first acceleration spectrum. The first average speed, the first speed standard deviation, and the first acceleration spectrum are combined to obtain a first motion feature matrix.

[0122] The second motion characteristic matrix is ​​used to represent the second motion characteristic matrices calculated for the second aircraft flight data collected using different acquisition methods. The second aircraft positioning data includes the second initial positioning data of the current aircraft entering the current airspace and the current second real-time positioning data of the aircraft. The second aircraft time data includes the second initial time data of the current aircraft entering the current airspace and the current second real-time time data of the aircraft. For each acquisition method, the second motion characteristic matrix corresponding to each acquisition method is calculated one by one through the following steps: calculating the second distance between the current second real-time positioning data of the aircraft and the second initial positioning data of the current aircraft entering the current airspace; calculating the second total time between the current second real-time time data of the aircraft and the second initial time data of the current aircraft entering the current airspace; dividing the second total distance by the second total time to obtain the second average speed; calculating the second speed standard deviation based on the second average speed; calculating the ratio between the second speed standard deviation and the second average speed to obtain the second speed coefficient of variation; calculating the second speed standard deviation based on the difference between the second aircraft speed data at different times and the second average speed; calculating the ratio between the second speed standard deviation and the second average speed to obtain the second speed coefficient of variation. A second acceleration sequence is obtained by differentiating the second aircraft velocity data and the second aircraft time data, and a second acceleration spectrum is constructed by combining the second angle data. The second average velocity, the second velocity standard deviation, and the second acceleration spectrum are combined to obtain a second motion feature matrix.

[0123] In this embodiment, a first motion feature matrix is ​​constructed based on first aircraft flight data collected by the aircraft itself to characterize the motion features of the first aircraft flight data collected by the aircraft itself. A second motion feature matrix is ​​constructed corresponding to each type of second aircraft flight data collected by a ground-based airspace management center to characterize the motion features of the second aircraft flight data collected by the ground-based airspace management center. This achieves a unified representation of multimodal motion features. Because the motion feature matrix is ​​used to characterize aircraft motion features, it remains unchanged after flight data transformation, facilitating subsequent feature evaluation using the motion feature matrix.

[0124] In one possible implementation, aircraft flight data from different sources are aligned in time and space to obtain aligned aircraft flight data, including:

[0125] Time-aligning the first aircraft time data and the second aircraft time data to obtain time-aligned first aircraft data and time-aligned second aircraft data;

[0126] Calculate the cosine similarity between the first motion feature matrix and the second motion feature matrix, construct a first spatial rotation matrix based on the cosine similarity, and spatially align the first aircraft flight data and the second aircraft flight data through the first spatial rotation matrix to obtain aligned first aircraft data and aligned second aircraft data, and use the aligned first aircraft data and the aligned second aircraft data as aligned aircraft data.

[0127] For the second aircraft time data collected using different acquisition methods, the following steps are performed to align the second aircraft time data collected using different acquisition methods: The DTW (Dynamic Time Warping) algorithm is used to align the second aircraft time data collected using different acquisition methods into the same time domain. The DTW algorithm is also used to align the first aircraft time data and the aligned second aircraft time data into the same time domain, resulting in the time-aligned first aircraft data and the time-aligned second aircraft data.

[0128] For the second aircraft data collected by different acquisition methods, the second aircraft data collected by different acquisition methods are spatially aligned pairwise through the following steps: the velocity vector of the second motion feature matrix of the current acquisition method at the current second aircraft flight time point and the velocity vector of the second motion feature matrix of the other acquisition method at the current second aircraft flight time point are obtained, and the cosine similarity between the velocity vector of the current acquisition method and the velocity vector of the other acquisition method at the current second aircraft flight time point is calculated. A second spatial rotation alignment matrix is ​​constructed, and the second aircraft flight data of the current acquisition method and the second aircraft flight data of the other acquisition method are aligned to the same airspace using the second spatial rotation matrix, wherein the second spatial rotation matrix is ​​used to represent the rotation matrix with the maximum cosine similarity of the second motion feature matrices at all second aircraft flight time points. After all second aircraft flight data are aligned to the same airspace, the first aircraft flight data and the aligned second aircraft flight data are aligned to the same airspace through the following steps: obtaining the velocity vector of the first motion feature matrix at the current aircraft flight time point and the velocity vector of the aligned second motion feature matrix at the current aircraft flight time point, and calculating the cosine similarity between the velocity vector of the first motion feature matrix at the current aircraft flight time point and the velocity vector of the aligned second motion feature matrix. Constructing a first spatial rotation matrix, and using the first spatial rotation matrix to align the first aircraft flight data and the aligned second aircraft flight data to the same airspace, wherein the first spatial rotation matrix is ​​used to represent the rotation matrix with the maximum cosine similarity between the first motion feature matrix and the second motion feature matrix at all aircraft flight time points.

[0129] In this embodiment, the trajectory data of different aircraft may have inconsistent time series due to various reasons. By using the DTW algorithm to align the time series and performing spatial alignment through the cosine similarity of the motion feature matrix, the aircraft data collected by different collection methods can be aligned, and the relationship between the aircraft flight data collected by different collection methods can be accurately compared and analyzed.

[0130] S204 : performing feature matching evaluation on the aligned aircraft flight data according to the motion feature matrix to obtain an aircraft flight data set belonging to the same aircraft.

[0131] In one possible implementation, feature matching evaluation is performed on the aligned aircraft flight data according to the motion feature matrix to obtain an aircraft flight data set belonging to the same aircraft, including:

[0132] Inputting the first motion feature matrix and the second motion feature matrix into a pre-trained random forest model, and outputting a feature matching degree between the first motion feature matrix and the second motion feature matrix;

[0133] The first motion feature matrix and the second motion feature matrix having a feature matching degree greater than a feature matching degree threshold are screened, and it is determined that the first aircraft flight data corresponding to the first motion feature matrix and the second aircraft flight data corresponding to the second motion feature matrix having a feature matching degree greater than the feature matching degree threshold belong to aircraft flight data of the same aircraft, and the first aircraft flight data and the second aircraft flight data belonging to the same aircraft are combined to obtain an aircraft flight data set belonging to the same aircraft.

[0134] In one example, the first motion feature matrix of the current aircraft is merged with each second motion feature matrix in the current airspace to obtain a joint feature vector set of the current aircraft; the current joint feature vector is input into a pre-trained random forest model to obtain a feature matching degree between the first motion feature matrix of the current aircraft and the current second motion feature matrix; the feature matching degree between the first motion feature matrix of the current aircraft and each second motion feature matrix in the current airspace is obtained one by one; the second motion feature matrix whose feature matching degree with the first motion feature matrix is ​​greater than a feature matching degree threshold is screened, and it is determined that the second aircraft flight data corresponding to the second motion feature matrix belongs to the current aircraft; the first aircraft flight data and the second aircraft flight data of the current aircraft are combined to obtain an aircraft flight data set of the current aircraft, wherein the aircraft flight data set of the current aircraft includes: a first aircraft identification, first aircraft time data, first aircraft positioning data, second aircraft time data, second aircraft positioning data, second aircraft speed data, and second aircraft angle data.

[0135] In this embodiment, since the first aircraft flight data is collected by the aircraft itself, a pre-trained random forest model is used to screen a second motion feature matrix whose feature matching degree with the first motion matrix of the current aircraft flight data is greater than a feature matching degree threshold, and the second aircraft flight data belonging to the current aircraft is determined. The aircraft flight data set belonging to the current aircraft is obtained by combining them, which facilitates the subsequent use of the aircraft flight data belonging to the same aircraft for flight trajectory fitting, further ensuring the accuracy of the flight trajectory fitting and reducing the amount of data calculation for the flight trajectory fitting.

[0136] S205: Determine the flight trajectory of the aircraft based on the aircraft flight data set belonging to the same aircraft.

[0137] In one possible implementation, before determining the flight trajectory of the aircraft based on the aircraft flight data set belonging to the same aircraft, the method further includes:

[0138] The mean and standard deviation of aircraft flight data belonging to the same aircraft from different sources are calculated using the isolation forest algorithm;

[0139] A threshold range is constructed based on the mean value and standard deviation, and aircraft flight data that does not fall within the threshold range is determined as an outlier. The aircraft flight data belonging to the same aircraft are cleaned of outliers to obtain the cleaned aircraft flight data, so as to determine the flight trajectory of the aircraft based on the cleaned aircraft flight data.

[0140] In one example, a first aircraft identifier of a current aircraft is determined as the aircraft identifier of the current aircraft. Using an isolation forest algorithm, the mean and standard deviation between the first aircraft positioning data of the current aircraft and all second aircraft positioning data belonging to the same aircraft are calculated. Based on the Laida criterion, a threshold range is constructed using the mean and standard deviation. A determination is made as to whether the first aircraft positioning data of the current aircraft and all second aircraft positioning data belonging to the same aircraft are within the threshold range. Aircraft positioning data outside the threshold range is determined as aircraft positioning data outliers, which are then filtered out to obtain cleaned aircraft positioning data.

[0141] Through the isolation forest algorithm, the mean and standard deviation between the first aircraft time data of the current aircraft and all the second aircraft time data belonging to the same aircraft are calculated. Based on the Laida criterion, the threshold range is constructed using the mean and standard deviation to determine whether the first aircraft time data of the current aircraft and all the second aircraft time data belonging to the same aircraft are within the threshold range. The aircraft time data outside the threshold range is determined as an aircraft time data anomaly point, and the aircraft time data anomaly points are filtered out to obtain the cleaned aircraft time data.

[0142] In this embodiment, the isolation forest algorithm has the characteristics of linear computational time complexity and low memory overhead. The isolation forest algorithm is used to remove aircraft flight data that are obviously outliers, thereby eliminating low-quality or erroneous aircraft flight data, improving the accuracy of the aircraft flight data used to fit the aircraft flight trajectory, and thus improving the aircraft flight trajectory fitting effect.

[0143] In one possible implementation, determining the flight trajectory of an aircraft based on a set of aircraft flight data belonging to the same aircraft includes:

[0144] Setting hash indexes of space-time cubes corresponding to first aircraft flight data and second aircraft flight data belonging to the same aircraft, respectively, and deduplicating the first aircraft flight data and the second aircraft flight data belonging to the same aircraft based on the hash indexes to obtain deduplicated aircraft flight data at multiple time points;

[0145] The Kalman filter algorithm is used to filter the deduplicated aircraft flight data at each time point to determine the target aircraft flight data;

[0146] When there is a lack of target aircraft flight data at the current time point, cubic spline interpolation is used to fill in the target aircraft flight data at the current time point until target aircraft flight data exists at each time point. The target aircraft flight data at each time point are combined to obtain fused aircraft flight data. The flight trajectory of the aircraft is determined based on the fused aircraft flight data.

[0147] In one example, the current airspace is divided into several three-dimensional grids based on longitude, latitude, and altitude, wherein the resolution of the three-dimensional grid is set according to the accuracy requirement and is not limited in this application. The time axis is divided into fixed windows. A corresponding unique hash index is generated for each first aircraft flight data and the second aircraft flight data, wherein the hash index is generated according to the longitude, latitude, altitude and time of the aircraft flight data. Based on the hash indexes corresponding to the first aircraft flight data and the second aircraft flight data, it is determined whether the first aircraft flight data and the second aircraft flight data are in the same space-time cube. When the first aircraft flight data and the second aircraft flight data are in the same space-time cube, it is determined that the aircraft flight data in the same space-time cube are space-time duplicates, and the aircraft flight data with the highest confidence or the highest data source priority is selected, and other redundant points are deleted to obtain deduplicated aircraft flight data for multiple time points. The method for determining the confidence and data source priority is not limited in this application.

[0148] The Kalman filter algorithm is used to estimate and predict the real-time operating status of the deduplicated aircraft flight data at each time point, and the predicted flight data corresponding to each time point is obtained; based on the estimated difference between the deduplicated aircraft flight data at the current time point and the corresponding predicted flight data, the target aircraft flight data at the current time point is determined.

[0149] When target aircraft flight data is missing at the current time point, a cubic spline function is constructed based on the flight data of aircraft at adjacent time points. This function is then used to interpolate the time points where target aircraft flight data is missing until target aircraft flight data is present at every time point. The infilled target aircraft flight data at each time point is combined to obtain fused flight data. This fused flight data is then used to fit the aircraft's flight trajectory.

[0150] In this embodiment, by setting a space-time cube and a corresponding hash index, redundant trajectory points caused by overlapping observations of multiple acquisition methods are eliminated; the asynchronous / heterogeneous aircraft flight data collected by different acquisition methods are fused through the Kalman filter algorithm to achieve the optimal estimation of the dynamic state of the aircraft; when there is a lack of target aircraft flight data at the current time point, cubic spline interpolation is used to generate a smooth trajectory that conforms to the kinematic laws to generate the aircraft's flight trajectory, further improving the aircraft flight trajectory fitting effect.

[0151] The multimodal information-based aircraft trajectory fitting method provided in the embodiments of the present application obtains flight data of an aircraft of different modes collected by multiple acquisition methods. The flight data of the aircraft of different modes can more comprehensively and accurately characterize the flight of the aircraft, facilitating the subsequent use of the flight data to fit the flight trajectory. By screening the flight data, flight data that does not belong to the aircraft is filtered out, reducing the flight data required for fitting the flight trajectory, and further improving the speed and accuracy of flight trajectory fitting. By performing spatiotemporal alignment on flight data from different sources, flight data of different modes are standardized to the same spatiotemporal time, avoiding inaccurate flight data fitting caused by inconsistencies between flight data of different modes. Using a motion feature matrix, flight data of aircraft belonging to the same aircraft are aggregated to distinguish flight data of different aircraft, facilitating the subsequent use of flight data belonging to the same aircraft for flight trajectory fitting. The aircraft data of the same aircraft are fused to determine the flight trajectory of the aircraft. The difference between the flight trajectory of the unmanned aerial vehicle (UAV) fitted with flight data of different modes collected by multiple acquisition methods and the actual flight trajectory of the UAV is small, thereby improving the UAV flight trajectory fitting effect and thereby improving the monitoring effect of the UAV.

[0152] Figure 3This is a schematic diagram of the structure of the aircraft trajectory fitting device based on multimodal information provided by this application, such as Figure 3 As shown, the aircraft trajectory fitting device based on multimodal information provided in this embodiment includes:

[0153] The data acquisition module 301 is used to acquire flight data of flying objects from different sources in the current airspace. Different sources are used to represent different acquisition methods, and different acquisition methods correspond to different modalities. The flying object includes at least one moving object in the air.

[0154] The data screening module 302 is used to screen the flight data, filter out the flight data that does not belong to the aircraft, and obtain the aircraft flight data;

[0155] The feature matrix construction module 303 is used to construct a motion feature matrix of the corresponding source based on the aircraft flight data from different sources, and align the aircraft flight data from different sources in time and space to obtain aligned aircraft flight data;

[0156] A feature matching evaluation module 304 is configured to perform feature matching evaluation on the aligned aircraft flight data according to the motion feature matrix to obtain an aircraft flight data set belonging to the same aircraft;

[0157] The trajectory determination module 305 is configured to determine the flight trajectory of an aircraft based on a set of aircraft flight data belonging to the same aircraft.

[0158] In a possible implementation, the flight data includes the first aircraft data and the flight data of all flying objects in the current airspace; the data acquisition module 301 is specifically configured to:

[0159] The first aircraft data of the aircraft itself is collected by a sensor provided on the aircraft, the first aircraft data including: first aircraft positioning data, first aircraft identification and first aircraft time data;

[0160] The flight data of all flying objects in the current airspace are collected through the airspace management center set up on the ground, where the flight data include: flight time data, flight positioning data, flight speed data and flight angle data; the airspace management center includes at least: several 5G-A base stations, ADS-B systems, integrated fusion ground surveillance base stations, different types of sensors and radar systems, and the integrated fusion ground surveillance base stations are used to represent ground base stations with aircraft monitoring functions.

[0161] In one possible implementation, the data screening module 302 is specifically configured to:

[0162] Divide the current airspace into several sub-airspaces and generate a geographic location code list corresponding to each sub-airspace;

[0163] Insert each flying object's positioning data into the spatial index and combine it with the geographic location code list to obtain the trajectory bounding box corresponding to the flying object's positioning data;

[0164] Calculate the correlation between different trajectory bounding boxes;

[0165] Determine the flight data of the flying object that does not belong to the aircraft based on the correlation;

[0166] The flight data of the flying object that does not belong to the aircraft is filtered to obtain second aircraft data, and the second aircraft data is combined with the first aircraft data to obtain aircraft flight data, wherein the second aircraft data includes: second aircraft time data, second aircraft positioning data, second aircraft speed data, and second aircraft angle data.

[0167] In one possible implementation, the motion feature matrix includes average velocity, velocity variation coefficient, and acceleration spectrum;

[0168] The feature matrix construction module 303 is specifically used for:

[0169] Based on the first aircraft positioning data and the first aircraft time data, a first average velocity, a first velocity variation coefficient and a first acceleration spectrum are calculated to construct a first motion feature matrix;

[0170] Based on the second aircraft time data, the second aircraft positioning data, the second aircraft speed data and the second aircraft angle data, a second average speed, a second speed variation coefficient and a second acceleration spectrum are calculated to construct a second motion feature matrix.

[0171] In a possible implementation, the feature matrix construction module 303 is further specifically configured to:

[0172] Time-aligning the first aircraft time data and the second aircraft time data to obtain time-aligned first aircraft data and time-aligned second aircraft data;

[0173] Calculate the cosine similarity between the first motion feature matrix and the second motion feature matrix, construct a first spatial rotation matrix based on the cosine similarity, and spatially align the first aircraft flight data and the second aircraft flight data through the first spatial rotation matrix to obtain aligned first aircraft data and aligned second aircraft data, and use the aligned first aircraft data and the aligned second aircraft data as aligned aircraft data.

[0174] In a possible implementation, the feature matching evaluation module 304 is specifically configured to:

[0175] Inputting the first motion feature matrix and the second motion feature matrix into a pre-trained random forest model, and outputting a feature matching degree between the first motion feature matrix and the second motion feature matrix;

[0176] The first motion feature matrix and the second motion feature matrix having a feature matching degree greater than a feature matching degree threshold are screened, and it is determined that the first aircraft flight data corresponding to the first motion feature matrix and the second aircraft flight data corresponding to the second motion feature matrix having a feature matching degree greater than the feature matching degree threshold belong to aircraft flight data of the same aircraft, and the first aircraft flight data and the second aircraft flight data belonging to the same aircraft are combined to obtain an aircraft flight data set belonging to the same aircraft.

[0177] In a possible implementation, the trajectory determination module 305 is specifically configured to:

[0178] Setting hash indexes of space-time cubes corresponding to first aircraft flight data and second aircraft flight data belonging to the same aircraft, respectively, and deduplicating the first aircraft flight data and the second aircraft flight data belonging to the same aircraft based on the hash indexes to obtain deduplicated aircraft flight data at multiple time points;

[0179] The Kalman filter algorithm is used to filter the deduplicated aircraft flight data at each time point to determine the target aircraft flight data;

[0180] When there is a lack of target aircraft flight data at the current time point, cubic spline interpolation is used to fill in the target aircraft flight data at the current time point until target aircraft flight data exists at each time point. The target aircraft flight data at each time point are combined to obtain fused aircraft flight data. The flight trajectory of the aircraft is determined based on the fused aircraft flight data.

[0181] In a possible implementation, the aircraft trajectory fitting device based on multimodal information is further specifically used for:

[0182] The mean and standard deviation of aircraft flight data belonging to the same aircraft from different sources are calculated using the isolation forest algorithm;

[0183] A threshold range is constructed based on the mean value and standard deviation, and aircraft flight data that does not fall within the threshold range is determined as an outlier. The aircraft flight data belonging to the same aircraft are cleaned of outliers to obtain the cleaned aircraft flight data, so as to determine the flight trajectory of the aircraft based on the cleaned aircraft flight data.

[0184] The aircraft trajectory fitting device based on multimodal information provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.

[0185] Figure 4 This is a schematic diagram of the structure of the electronic device provided in this application. Figure 4 As shown, the electronic device 40 provided in this embodiment includes: at least one processor 401 and a memory 402. Optionally, the device 40 further includes a communication component 403. The processor 401, the memory 402 and the communication component 403 are connected via a bus 404.

[0186] In a specific implementation process, at least one processor 401 executes the computer-executable instructions stored in the memory 402, so that the at least one processor 401 performs the above method.

[0187] The specific implementation process of the processor 401 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0188] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASICs), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly executed by a hardware processor or by a combination of hardware and software modules within the processor.

[0189] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage.

[0190] A bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be categorized as address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0191] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0192] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.

[0193] The readable storage medium may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0194] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0195] The division of units is merely a logical functional division; actual implementations may employ alternative divisions, such as combining or integrating multiple units or components into another system, or omitting or disabling certain features. Furthermore, any direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units, either through an interface, electrical, mechanical, or other means.

[0196] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0197] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0198] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0199] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0200] Finally, it should be noted that those skilled in the art will readily identify other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed herein. The present invention is not limited to the precise structure described above and illustrated in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof. The scope of the present invention is limited solely by the appended claims.

Claims

1. A method for fitting aircraft trajectories based on multimodal information, characterized in that: include: Acquire flight data of flying objects from different sources within the current airspace, where the different sources are used to represent different acquisition methods, and the different acquisition methods correspond to different modalities; wherein the flying object includes at least one moving object in the air; screening the flight data, filtering out flight data that does not belong to the aircraft, and obtaining aircraft flight data; Based on the aircraft flight data from different sources, a motion feature matrix of the corresponding source is constructed, and the aircraft flight data from different sources are aligned in time and space to obtain the aligned aircraft flight data; performing feature matching evaluation on the aligned aircraft flight data according to the motion feature matrix to obtain an aircraft flight data set belonging to the same aircraft; The flight trajectory of the aircraft is determined based on the aircraft flight data set belonging to the same aircraft.

2. The method according to claim 1, characterized in that The flight data includes the first aircraft data and the flight data of all flying objects in the current airspace; and obtaining the flight data of flying objects from different sources in the current airspace includes: The first aircraft data of the aircraft itself is collected by a sensor provided on the aircraft, wherein the first aircraft data includes: first aircraft positioning data, first aircraft identification, and first aircraft time data; The flight data of all flying objects in the current airspace are collected through an airspace management center set up on the ground, wherein the flight data include: flight time data, flight positioning data, flight speed data and flight angle data; wherein, the airspace management center includes at least: several 5G-A base stations, ADS-B systems, integrated fusion ground surveillance base stations, sensors and radar systems, and the integrated fusion ground surveillance base station is used to represent a ground base station with aircraft monitoring function.

3. The method according to claim 2, characterized in that The step of screening the flight data to filter out flight data that does not belong to the aircraft and obtaining the aircraft flight data includes: Divide the current airspace into a number of sub-airspaces, and generate a list of geographic location codes corresponding to each of the sub-airspaces; Inserting each of the aircraft positioning data into a spatial index and combining it with the geographic location code list to obtain a trajectory bounding box corresponding to the aircraft positioning data; Calculating the correlation between different trajectory bounding boxes; Determining flight data of an aircraft that does not belong to the aircraft based on the correlation; The flight data of the flying object that does not belong to the aircraft is filtered to obtain second aircraft data, and the second aircraft data and the first aircraft data are combined to obtain aircraft flight data, wherein the second aircraft data includes: second aircraft time data, second aircraft positioning data, second aircraft speed data and second aircraft angle data.

4. The method according to claim 3, characterized in that The motion characteristic matrix includes average velocity, velocity variation coefficient and acceleration spectrum; The method of constructing a motion feature matrix of a corresponding source based on aircraft flight data from different sources includes: Based on the first aircraft positioning data and the first aircraft time data, a first average velocity, a first velocity variation coefficient, and a first acceleration spectrum are calculated to construct a first motion feature matrix; Based on the second aircraft time data, the second aircraft positioning data, the second aircraft speed data and the second aircraft angle data, a second average speed, a second speed variation coefficient and a second acceleration spectrum are calculated to construct a second motion feature matrix.

5. The method according to claim 4, characterized in that The aircraft flight data from different sources are aligned in time and space to obtain aligned aircraft flight data, including: performing time alignment on the first aircraft time data and the second aircraft time data to obtain time-aligned first aircraft data and time-aligned second aircraft data; Calculate the cosine similarity between the first motion feature matrix and the second motion feature matrix, construct a first spatial rotation matrix based on the cosine similarity, and spatially align the first aircraft flight data and the second aircraft flight data using the first spatial rotation matrix to obtain aligned first aircraft data and aligned second aircraft data, and use the aligned first aircraft data and aligned second aircraft data as aligned aircraft data.

6. The method according to claim 5, characterized in that The step of performing feature matching evaluation on the aligned aircraft flight data according to the motion feature matrix to obtain an aircraft flight data set belonging to the same aircraft includes: Inputting the first motion feature matrix and the second motion feature matrix into a pre-trained random forest model, and outputting a feature matching degree between the first motion feature matrix and the second motion feature matrix; The first motion feature matrix and the second motion feature matrix having a feature matching degree greater than a feature matching degree threshold are screened, and it is determined that the first aircraft flight data corresponding to the first motion feature matrix and the second aircraft flight data corresponding to the second motion feature matrix having a feature matching degree greater than the feature matching degree threshold belong to aircraft flight data of the same aircraft, and the first aircraft flight data and the second aircraft flight data belonging to the same aircraft are combined to obtain an aircraft flight data set belonging to the same aircraft.

7. The method according to claim 6, characterized in that Determining the flight trajectory of an aircraft based on the aircraft flight data set belonging to the same aircraft includes: Setting hash indexes of the space-time cubes corresponding to the first aircraft flight data and the second aircraft flight data belonging to the same aircraft, respectively, and deduplicating the first aircraft flight data and the second aircraft flight data belonging to the same aircraft based on the hash indexes to obtain deduplicated aircraft flight data at multiple time points; The Kalman filter algorithm is used to filter the deduplicated aircraft flight data at each time point to determine the target aircraft flight data; When there is a lack of target aircraft flight data at the current time point, cubic spline interpolation is used to fill in the target aircraft flight data at the current time point until target aircraft flight data exists at each time point. The target aircraft flight data at each time point are combined to obtain fused aircraft flight data. The flight trajectory of the aircraft is determined based on the fused aircraft flight data.

8. The method according to claim 1, characterized in that Before determining the flight trajectory of an aircraft based on the aircraft flight data set belonging to the same aircraft, the method further includes: Calculating the mean and standard deviation of the aircraft flight data belonging to the same aircraft from different sources by using an isolation forest algorithm; A threshold range is constructed based on the mean value and the standard deviation, aircraft flight data that does not fall within the threshold range is determined as an outlier, and the aircraft flight data belonging to the same aircraft are cleaned of the outliers to obtain cleaned aircraft flight data, so as to determine the flight trajectory of the aircraft based on the cleaned aircraft flight data.

9. An aircraft trajectory fitting device based on multimodal information, characterized in that: include: A data acquisition module is configured to acquire flight data of flying objects from different sources within the current airspace, wherein the different sources are used to represent different acquisition methods, and the different acquisition methods correspond to different modalities; wherein the flying object includes at least one moving object in the air; a data screening module, configured to screen the flight data, filter out flight data that does not belong to the aircraft, and obtain aircraft flight data; A feature matrix construction module is used to construct motion feature matrices of the corresponding sources based on aircraft flight data from different sources, and to align the aircraft flight data from different sources in time and space to obtain aligned aircraft flight data; a feature matching evaluation module, configured to perform feature matching evaluation on the aligned aircraft flight data according to the motion feature matrix to obtain an aircraft flight data set belonging to the same aircraft; The trajectory determination module is used to determine the flight trajectory of the aircraft based on the aircraft flight data set belonging to the same aircraft.

10. An electronic device, characterized in that: include: Memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor performs the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 8 when executed by a processor.

Images