An LSTM-based multi-modal data fusion UAV positioning calibration method

Abstract

The application provides a kind of multi-modal data fusion unmanned plane positioning calibration method based on LSTM, it is related to unmanned plane navigation technical field.First, the original GPS signal of unmanned plane and multi-modal sensor data are acquired;The original GPS signal and multi-modal sensor data are segmented by using sliding window strategy;Take the time domain transient characteristics and frequency domain steady state characteristics of multi-modal sensor data, through feature concatenation and normalization operation, finally through the fusion of robust standard layer, feature attention layer, multi-layer LSTM constructs LSTM time series prediction model, takes RTK positioning result as true value, using the optimized LSTM time series prediction model, through the fusion of multiple modal sensor data to regress and predict RTK data, when RTK value is missing, use the fusion data to replace RTK true value to obtain accurate positioning, to complete unmanned plane positioning calibration work.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) navigation technology, and in particular to a UAV positioning and calibration method based on LSTM multimodal data fusion. Background Technology

[0002] In the process of intelligent transformation of the mining production chain, drones, with their advantages of high efficiency, flexibility, and low cost, have been widely used in many key scenarios such as geological exploration, topographic mapping, resource reserve estimation, and mine safety monitoring. However, existing drone mapping technology faces significant challenges in complex mining environments, with its core bottlenecks mainly lying in positioning accuracy and reliability. Mining areas often have extremely complex terrains, such as deep canyons, densely vegetated areas, and open-pit slopes. These environmental characteristics easily cause Global Navigation Satellite System (GNSS) signals, especially Real-Time Kinematic (RTK) measurement signals, to be interfered with by multipath effects or directly blocked and interrupted. When RTK signals fail, relying solely on a single sensor such as an Inertial Measurement Unit (IMU) for positioning results in errors that accumulate rapidly over time, leading to significant deviations in trajectory calculations from the true values. Ultimately, this makes it difficult to meet the stringent requirements for horizontal and vertical accuracy in large-scale (e.g., 1:2000 or even 1:500) high-precision topographic mapping.

[0003] To address the aforementioned challenges and improve the positioning robustness of UAVs in GNSS-denied environments, current research and practice mainly focus on two calibration and positioning correction paths:

[0004] The first approach is real-time dynamic calibration and fusion positioning. This method focuses on estimating and compensating for positioning errors in real time during UAV flight by fusing multi-source sensor data online. The most representative technical solution is tightly coupled or loosely coupled GNSS / INS (Inertial Navigation System) integrated navigation. When RTK signals are available, the system continuously corrects the accumulated errors of the IMU using high-precision position information. Once the RTK signal quality deteriorates or is lost, the system can immediately switch to using calibrated inertial sensor data as the primary source, supplemented by data from other sensors (such as visual odometry, lidar odometry, or barometers). Advanced estimation algorithms such as Extended Kalman Filter (EKF) or Unscented Kalman Filter (UKF) are used for autonomous inertial navigation and short-term trajectory prediction, thereby maintaining usable positioning accuracy to a certain extent and ensuring the continuity of flight and data acquisition tasks.

[0005] The second approach is post-processing calibration and data optimization. This method does not pursue absolute real-time high accuracy during flight. Instead, after the mission is completed, it performs offline depth processing on the raw sensor data and corrects the overall positioning information through optimization algorithms. This typically involves post-processing dynamic positioning based on smoothing algorithms, such as using the Rauch-Tung-Striebel smoother or a graph optimization framework. Within this framework, the raw IMU data, intermittently available GNSS observations, and relative pose constraints generated by visual feature point matching or laser point cloud matching are collectively constructed into an optimization problem. By minimizing the overall error, a globally consistent and better flight trajectory is obtained. Post-processing methods can fully utilize all available information and usually achieve higher absolute positioning accuracy than purely real-time systems. This is particularly suitable for mapping and modeling tasks where real-time requirements are not high, but the final result accuracy is extremely important.

[0006] While both approaches have their own focus and have achieved some progress, limitations remain in the complex environments of actual mines. The accuracy and reliability of real-time fusion systems are highly dependent on the quality of initial sensor calibration, and errors can still diverge under prolonged GNSS rejection conditions. Post-processing methods suffer from processing delays and cannot support automated operations requiring real-time high-precision positioning (such as precise material delivery and autonomous inspection). Therefore, future research trends are moving towards a two-stage optimization framework combining real-time and post-processing, and towards incorporating artificial intelligence algorithms (such as deep learning) for sensor error modeling and adaptive environmental filtering, aiming to build a more robust and accurate high-precision positioning solution for mine drones. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention provides a UAV positioning and calibration method based on LSTM multimodal data fusion. Through multimodal feature extraction and data fusion algorithms, it achieves high-precision positioning correction for UAVs, supporting precise operations in scenarios such as geological exploration and mining development.

[0008] On the one hand, this invention provides a UAV positioning and calibration method based on LSTM multimodal data fusion, comprising the following steps:

[0009] Step 1: Acquire the drone's raw GPS signal and multimodal sensor data;

[0010] Step 2: Use a sliding window strategy to segment the raw GPS signal and multimodal sensor data;

[0011] Specifically, the window length is set to 30 sampling points, and the sliding step is 15 sampling points, meaning that there are 15 historical and 15 future data points at the beginning and end.

[0012] Step 3: Input the multimodal sensor data sequence in the window into the time-frequency feature extraction module;

[0013] The time-frequency feature extraction module extracts the transient time-domain features and steady-state frequency-domain features from the multimodal sensor data, and constructs a multidimensional time-frequency joint feature vector through feature concatenation and normalization operations. , where d is the feature dimension;

[0014] Step 4: Construct an LSTM time series prediction model that incorporates an attention mechanism;

[0015] The LSTM time-series prediction model consists of a robust standard layer, a feature attention layer, and a multi-layer LSTM.

[0016] The robust standardization layer uses interquartile range to robustly standardize the data and remove extreme outliers, as shown in the following formula:

[0017] IQR = Q3 - Q1 (1);

[0018] Where Q1 is the first quartile and Q3 is the third quartile, only the data between these two quartiles are retained;

[0019] The feature attention layer is based on a multilayer perceptron (MLP) and is used to dynamically analyze the importance of different input features and calculate the importance weights. The specific process is as follows:

[0020] First, perform feature transformation: (2);

[0021] Where X represents the extracted features, W1 and b1 are trainable parameters, σ(•) is the sigmoid activation function, and f θ (•) is the linear transformation function of the characteristic transformation;

[0022] Attention function: (3);

[0023] Let be the attention function. Given n×d dimensional data, the n×d dimensions are mapped to n×n dimensions, A is the attention weight, and softmax(•) is the softmax activation function. It is an MLP function, where d is the dimension, Q is the query matrix, and K is the key matrix.

[0024] The output after weighting the feature data with attention parameters is: (4); among which This is a transformation function that combines linear transformation with attention weights for weighting;

[0025] Final output prediction: (5);

[0026] Both W2 and b2 are trainable parameters, and Concat(•) is the concatenation function.

[0027] The multilayer LSTM includes several gate structures, as detailed below;

[0028] First, through the forget gate, the hidden state h from the previous time step is input. t-1 and the input x at the current time step t , and trainable parameters W f U f b f Calculations are performed to determine the forgotten content, as shown in formula (6):

[0029] (6);

[0030] Then, the input gate is used, by inputting the hidden state h from the previous time step. t-1 and the input x at the current time step t , and trainable parameters W i U i b i Calculations are performed to determine the content to be memorized, as shown in formula (7):

[0031] (7);

[0032] Then, the cell state is updated, which is divided into two stages. One stage is to calculate the candidate state, as shown in formula (8), by updating the hidden state h of the previous time step. t-1 and the input x at the current time step t After linear transformation, activation is performed using the activation function tanh(•); one is to update the cell state, as shown in formula (9), by weighting and summing the candidate state, forget gate, and input gate to train the model parameters and achieve the effect of fitting the function:

[0033] (8);

[0034] (9);

[0035] Among them, W c U c b c represents trainable parameters, and ⊙ represents element-wise multiplication;

[0036] Finally, the output gate and the current hidden state are used as the output to provide parameters for the next LSTM update. The output gate is shown in formula (10), and the current hidden state is output by formula (11):

[0037] (10);

[0038] (11);

[0039] Among them, W0, U0, and b0 are trainable parameters;

[0040] For activation function: (12);

[0041] Step 5: Optimize the LSTM time series prediction model;

[0042] Root Mean Square Error (RMSE) is used to quantify the difference between the model's predicted values ​​and the actual values. The calculation formula is as follows:

[0043] (13);

[0044] Among them, y i It is the actual value. is the predicted value, and n is the number of samples.

[0045] The LSTM time series prediction model is optimized by minimizing the RMSE.

[0046] Step 6: Using the RTK positioning result as the ground truth, the optimized LSTM time series prediction model is used to regress and predict the RTK data by fusing sensor data from multiple modalities. When the RTK value is missing, the fused data is used to replace the ground truth RTK value to obtain accurate positioning, thereby completing the UAV positioning calibration work.

[0047] On the other hand, this application proposes a computer-readable storage medium storing executable instructions that, when executed, cause a processor to perform the aforementioned LSTM-based multimodal data fusion UAV positioning and calibration method.

[0048] Thirdly, this application proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned LSTM-based multimodal data fusion UAV positioning and calibration method.

[0049] The beneficial effects of adopting the above technical solution are as follows:

[0050] This invention provides a UAV positioning calibration method based on LSTM multimodal data fusion. The method generates denser contour lines after calibration, significantly improving positioning accuracy. This method effectively solves the positioning robustness problem in complex environments with GNSS denial scenarios, overcoming the limitations of existing real-time calibration and post-processing methods, and providing reliable technical support for high-precision UAV operations. Experiments show that before calibration, the RMSE of GPS and RTK both exceeded 100cm in latitude, longitude, and altitude. After calibration, the RMSE differences in latitude, longitude, and altitude were 5.0685cm, 9.4284cm, and 14.3908cm, respectively, demonstrating a significant improvement in accuracy. Attached Figure Description

[0051] Figure 1 Structure diagram of the LSTM time series prediction model in this embodiment of the invention;

[0052] Figure 2 Multilayer LSTM structure diagram of an embodiment of the present invention;

[0053] Figure 3 Error distribution diagram of extracted_data in an embodiment of the present invention;

[0054] Where (a) is the longitude error map after calibration, (b) is the latitude error map after calibration, and (c) is the altitude error map after calibration;

[0055] Figure 4 Error distribution diagram of embodiment 2025-02-11_11-04-21_FLY089 of the present invention;

[0056] Where (a) is the longitude error map after calibration, (b) is the latitude error map after calibration, and (c) is the altitude error map after calibration.

[0057] Figure 5 Comparison of data before and after calibration in an embodiment of the present invention;

[0058] Where (a) is the image before calibration, and (b) is the image after calibration. Detailed Implementation

[0059] The specific implementation methods of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0060] Example 1:

[0061] This invention provides a UAV positioning and calibration method based on LSTM multimodal data fusion, comprising the following steps:

[0062] Step 1: Acquire the drone's raw GPS signal and multimodal sensor data;

[0063] In this embodiment, the DJI RC Plus remote controller is turned on, and the DJI RC Plus is connected to the computer using a USB-C data cable. Specifically, flight control data is obtained from the DJI RC Plus remote controller. The remote controller and computer are connected using a USB-C data cable, FlightRecord data is read from the remote controller, and then the data is input into the DatCon data processing software. After selecting the corresponding processing parameters, multimodal data can be obtained. In this embodiment, VIO, GPS, barometric pressure, speed and other sensor data are used.

[0064] Step 2: To capture the temporal correlation and dynamic characteristics of sensor signals, a sliding window strategy is used to segment the raw GPS signal and multimodal sensor data.

[0065] Specifically, the window length is set to 30 sampling points (corresponding to a time span of 6 seconds and a sampling rate of 5 Hz in this embodiment), and the sliding step size is 15 sampling points. Taking the data at time t=15 as an example, the window coverage range is from t-14≤t≤t+15, that is, it contains 15 historical and future data points before and after. This design can balance computational efficiency and time continuity, while avoiding information redundancy caused by excessive window overlap.

[0066] Step 3: Input the multimodal sensor data sequence in the window into the time-frequency feature extraction module;

[0067] The time-frequency feature extraction module extracts the time-domain transient features (such as acceleration abrupt changes and angular velocity oscillations) and frequency-domain steady-state features (such as low-frequency trends of air pressure drift and high-frequency noise of electromagnetic interference) from the multimodal sensor data. Through feature concatenation and normalization operations, a multidimensional time-frequency joint feature vector is constructed. , where d is the feature dimension;

[0068] In this embodiment, a total of 28 time-frequency domain features are extracted for the time-series signals of each type of sensor, including 15 types of time-domain features and 13 types of frequency-domain features. The 15 time-domain statistical indicators are shown in Table 1.

[0069] Table 1. Time-domain characteristics:

[0070] Where x(n) is the time-domain sequence of the signal, n=1,2,…,N, and N is the number of sample points. The above time-domain statistical characteristics can be directly calculated from the time-series data of the vibration signal, as shown in the formulas in Table 1, where: P1, P3-P5 reflect the time-domain amplitude and energy characteristics of the signal; P2, P6-P 11 P reflects the time-domain distribution characteristics of the vibration signal. 12It measures the sharpness of the vibration signal waveform; P 13 P represents the ratio of the peak value to the root mean square value of the vibration signal; 14 -P 15 It reflects the distribution of vibration signals.

[0071] The formulas for the other 13 frequency domain indices are shown in Table 2.

[0072] Table 2 Frequency Domain Characteristics:

[0073] Where s(k) is the spectrum of the signal, k = 1, 2, ..., K, and K is the number of spectral lines; f k This is the frequency value of the k-th spectral line. F1 reflects the frequency domain amplitude and energy characteristics of the signal, while F2-F4, F6, and F... 10 -F 12 This reflects the frequency domain distribution characteristics of the signal; F5 and F7-F9 reflect the main frequency peaks of the vibration signal, F 13 This reflects the standard deviation frequency of the vibration signal;

[0074] Through the above calculations, the time-series signal of each sensor can be represented as 28 features.

[0075] Step 4: Construct an LSTM time series prediction model that incorporates an attention mechanism;

[0076] The LSTM temporal prediction model sequentially comprises a robust standard layer, a feature attention layer, and a multi-layer LSTM; its structure is as follows: Figure 1 As shown, each layer makes an outstanding contribution to the model output.

[0077] The robust standardization layer uses interquartile range to robustly standardize the data, removing extreme outliers, making the model more closely resemble actual flight paths during training. The formula is as follows:

[0078] IQR = Q3 - Q1 (1);

[0079] Where Q1 is the first quartile and Q3 is the third quartile, only the data between these two quartiles are retained;

[0080] The feature attention layer is based on a multilayer perceptron (MLP) and is used to dynamically analyze the importance of different input features during training and calculate importance weights, thereby amplifying important feature values ​​and reducing irrelevant feature values. Then, it is input into an LSTM to obtain more accurate predictions. The specific process is as follows:

[0081] First, perform feature transformation: (2);

[0082] Where X represents the extracted features, W1 and b1 are trainable parameters, σ(•) is the sigmoid activation function, and f θ (•) is the linear transformation function of the feature transformation.

[0083] Attention function: (3);

[0084] Let be the attention function. Given n×d dimensional data, the n×d dimensions are mapped to n×n dimensions, A is the attention weight, and softmax(•) is the softmax activation function. It is an MLP function, where d is the dimension, Q is the query matrix, and K is the key matrix.

[0085] The output after weighting the feature data with attention parameters is: (4); among which This is a transformation function that combines linear transformation with attention weights for weighting;

[0086] Final output prediction: (5);

[0087] Both W2 and b2 are trainable parameters, and Concat(•) is the concatenation function.

[0088] The multilayer LSTM, such as Figure 2 As shown, it is a widely validated recurrent neural network (RNN). RNNs can be used to capture temporal dependencies, while LSTM is specifically designed to solve the gradient vanishing and long-term dependency problems of traditional RNNs when processing long sequence data. It contains several gate structures, as follows.

[0089] First, a forget gate is used to determine which information to discard from the previous cell state, by inputting the hidden state h from the previous time step. t-1 and the input x at the current time step t , and trainable parameters W f U f b f Calculations are performed to determine the forgotten content, as shown in formula (6):

[0090] (6);

[0091] Then the input gate is used, which determines which new information to store in the cell state by inputting the hidden state h from the previous time step. t-1 and the input x at the current time step t , and trainable parameters W i U i b iCalculations are performed to determine the content to be memorized, as shown in formula (7):

[0092] (7);

[0093] Then, the cell state is updated. This is done by determining which memories need to be forgotten and which memories need to be remembered. This is done in two stages: one is to calculate the candidate state, as shown in formula (8), by updating the hidden state h from the previous time step. t-1 and the input x at the current time step t After linear transformation, activation is performed using the activation function tanh(•); one is to update the cell state, as shown in formula (9), by weighting and summing the candidate state, forget gate, and input gate, and by training the model parameters through the above calculation method to achieve the effect of fitting the function:

[0094] (8);

[0095] (9);

[0096] Among them, W c U c b c represents trainable parameters, and ⊙ represents element-wise multiplication;

[0097] Finally, the output gate and the current hidden state are used as the output to provide parameters for the next LSTM update. The output gate is shown in formula (10), and the current hidden state is output by formula (11):

[0098] (10);

[0099] (11);

[0100] Among them, W0, U0, and b0 are trainable parameters;

[0101] Following the multi-layer LSTM are fully connected layers. Using four methods—dimensionality compression and expansion, nonlinear modeling, regularization mechanisms, and parameter control—efficient feature transformation is achieved through a "compression-activation-expansion" pipeline.

[0102] For activation function: (12);

[0103] In this embodiment, the above-mentioned layers are integrated to form a multi-layer LSTM with a fusion attention mechanism used in this study. The hyperparameters set during training are as follows:

[0104] The input dimension is the number of columns in the feature matrix obtained after feature extraction, and the output dimension consists of three columns: latitude, longitude, and elevation.

[0105] The hidden layer has a dimension of 256;

[0106] The probability of dropout is set to 0.9;

[0107] The dropout value for attention is 0.3;

[0108] Robust standardization will be used;

[0109] The parameters after starting model training are:

[0110] Use MSE as the loss function;

[0111] Use GPUs for model training;

[0112] The Adam optimizer and StepLR learning rate scheduler are used to optimize the model training process.

[0113] Use gradient clipping to prevent gradient explosion;

[0114] Step 5: Optimize the LSTM time series prediction model;

[0115] To evaluate the model's performance, the root mean square error (RMSE) was chosen as the primary evaluation metric. RMSE quantifies the difference between the model's predicted values ​​and the actual values, and its calculation formula is as follows:

[0116] (13);

[0117] Among them, y i It is the actual value. is the predicted value, and n is the number of samples.

[0118] By optimizing the LSTM time series prediction model by minimizing the RMSE, the prediction accuracy of the model can be effectively improved.

[0119] In this embodiment, five time-series prediction models were compared: RANSAC regression, linear regression, support vector regression (SVR), random forest regression, and LSTM with an attention mechanism. RANSAC regression was used to handle outliers in the data, setting at least 80% of the samples as inliers, a residual threshold of 3.0, and a maximum of 100 iterations. The linear regression model did not adjust hyperparameters. For SVR, a linear kernel function was used, with the regularization parameter C set to 100 and epsilon set to 0.001. The random forest regression model constructed 200 decision trees, each with a maximum depth of 20, considering 40% of the features in each split, and using random seeds to ensure reproducibility. All models were accelerated using multi-core CPUs. The selection of these parameters comprehensively considered model performance, computational efficiency, and generalization ability. Experimental results show that the optimized model achieved a low RMSE on the test set, validating the model's effectiveness and robustness.

[0120] Step 6: Using the RTK (Real-Time Kinematic) positioning result as the ground truth, the optimized LSTM time-series prediction model is used to regress and predict the RTK data by fusing sensor data from multiple modalities. When the RTK value is missing, the fused data is used to replace the RTK ground truth to obtain accurate positioning, thereby completing the UAV positioning calibration work.

[0121] Example 2:

[0122] The study area is an actual mining area. Its GNSS data includes latitude, longitude, elevation, and triaxial velocity features, while its RTK data includes latitude, longitude, and elevation features. Both data have a sampling rate of 5Hz. In addition to the aforementioned sensors, data features from sensors such as IMU and VIO, including velocity, acceleration, and barometer altitude, are also included. The Earth is an ellipsoid, slightly flattened at the poles and bulging at the equator. Therefore, the curvature of the Earth's surface varies at different latitudes, which necessitates taking into account the Earth's ellipsoidal shape when calculating distances between latitudes and longitudes.

[0123] The random forest algorithm performed relatively well, with overall root mean square errors all less than 20 cm. The method based on LTSM and attention showed the best performance, demonstrating that the proposed method has good performance and practical value.

[0124] Table 3 Experimental Results: Linear Regression 39.5634 16.0621 43.869 RANSAC 61.8281 25.7277 48.2099 Random Forest 7.6838 14.9000 16.3294 LSTM + Attention 5.0685 9.4284 14.3908

[0125] This project requires high-precision calibration. To address the issue of varying error distributions across different regions, the Earth is divided into projection zones. These zones are divided from west to east, with each zone representing a 3-degree difference in longitude, resulting in a total of 120 projection zones globally. The algorithm's performance is verified on two routes within the same projection zone: route 1 named "2025-02-11_11-04-21_FLY089" and route 2 named "2025-02-11_11-49-37_FLY090_200hz".

[0126] Table 4. Calibration errors for different flight paths within the same projection zone (unit: cm): Route 1 6.6465 2.3126 9.4843 Route 2 5.0685 9.4284 14.3908

[0127] As can be seen from the table above, the error differences between different routes are small, indicating that the algorithm is transferable within the same projection zone and meets the actual needs of the project.

[0128] The test was conducted using actual data from the Shihu area. The results of the data designed in this patent after standardization are as follows: Figure 3 , Figure 4 As shown, where, Figure 3 as well as Figure 4 (a), (b), and (c) represent the longitude error map, latitude error map, and altitude error map after calibration, respectively. It can be seen that the error limits for longitude, latitude, and altitude are all less than 20cm, while the mean square error is within 10cm, and even as low as 5cm. This fully demonstrates the effectiveness of the algorithm in this patent and provides a new solution for UAV calibration.

[0129] After calibration, the calibrated data is used in subsequent data processing software results, such as... Figure 5 As shown, where Figure 5 (a) and (b) are the images before and after calibration, respectively. It can be seen that the contour lines in the calibrated image are denser and more accurate than the original data, which indirectly verifies the validity of this patent.

[0130] Example 3:

[0131] This embodiment proposes a computer-readable storage medium that stores executable instructions. When these instructions are executed, if they are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.

[0132] The computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the LSTM-based multimodal data fusion UAV positioning and calibration method described in the various embodiments of this application.

[0133] The aforementioned storage media include: flash memory, hard disk, multimedia card, card-type memory (e.g., SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory, random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, server, APP (Application) application store, and other media capable of storing program verification codes. These media store computer programs, and when executed by a processor, they can implement the various steps of the aforementioned LSTM-based multimodal data fusion UAV positioning and calibration method.

[0134] Example 4:

[0135] This embodiment proposes a computer program product, including a computer program or instructions, which, when executed by a processor, implements the aforementioned LSTM-based multimodal data fusion UAV positioning and calibration method.

[0136] Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or part of the technical solution, can be embodied in the form of a computer program product.

[0137] The various embodiments in this application are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0138] The scope of protection of this application is not limited to the embodiments described above. Obviously, those skilled in the art can make various modifications and variations to this disclosure without departing from the scope and spirit of this disclosure. If such modifications and variations fall within the scope of the methods disclosed herein and their equivalents, then the intent of this disclosure also includes such modifications and variations.

Claims

1. A UAV positioning and calibration method based on LSTM multimodal data fusion, characterized in that, Includes the following steps: Step 1: Acquire the drone's raw GPS signal and multimodal sensor data; Step 2: Use a sliding window strategy to segment the raw GPS signal and multimodal sensor data; Step 3: Input the multimodal sensor data sequence in the window into the time-frequency feature extraction module; Step 4: Construct an LSTM time-series prediction model with an integrated attention mechanism; the LSTM time-series prediction model sequentially includes a robust standard layer, a feature attention layer, and a multi-layer LSTM. Step 5: Optimize the LSTM time series prediction model; Step 6: Using the RTK positioning result as the ground truth, the optimized LSTM time series prediction model is used to regress and predict the RTK data by fusing sensor data from multiple modalities. When the RTK value is missing, the fused data is used to replace the ground truth RTK value to obtain accurate positioning, thereby completing the UAV positioning calibration work.

2. The UAV positioning and calibration method based on LSTM multimodal data fusion according to claim 1, characterized in that, Step 2 specifically involves setting the window length to 30 sampling points and the sliding step size to 15 sampling points, meaning that the front and back each contain 15 historical and future data points.

3. The UAV positioning and calibration method based on LSTM multimodal data fusion according to claim 1, characterized in that, The time-frequency feature extraction module described in step 3 extracts the transient time-domain features and steady-state frequency-domain features of the multimodal sensor data, respectively. Through feature concatenation and normalization operations, a multidimensional time-frequency joint feature vector is constructed. , where d is the feature dimension.

4. The UAV positioning and calibration method based on LSTM multimodal data fusion according to claim 3, characterized in that, The robust standardization layer described in step 4 uses interquartile range to robustly standardize the data and remove extreme outliers. The formula is as follows: IQR = Q3 - Q1 (1); Where Q1 is the first quartile and Q3 is the third quartile, only the data between these two quartiles are retained.

5. The UAV positioning and calibration method based on LSTM multimodal data fusion according to claim 4, characterized in that, The feature attention layer in step 4 is based on a multilayer perceptron (MLP) and is used to dynamically analyze the importance of different input features and calculate the importance weights. The specific process is as follows: First, perform feature transformation: (2); where X is the extracted features, W1, b1 are trainable parameters, σ(•) is a sigmoid activation function, f θ (•) is a linear transformation function of feature transformation; Attention function: (3); Let be the attention function. Given n×d dimensional data, the n×d dimensions are mapped to n×n dimensions, A is the attention weight, and softmax(•) is the softmax activation function. It is an MLP function, where d is the dimension, Q is the query matrix, and K is the key matrix; The output after weighting the feature data with attention parameters is: (4); among which This is a transformation function that combines linear transformation with attention weights for weighting; Final output prediction: (5); Both W2 and b2 are trainable parameters, and Concat(•) is the concatenation function.

6. The UAV positioning and calibration method based on LSTM multimodal data fusion according to claim 5, characterized in that, The multilayer LSTM includes several gate structures, as detailed below; First, through the forget gate, the hidden state h from the previous time step is input. t-1 and the input x at the current time step t , and trainable parameters W f U f b f Calculations are performed to determine the forgotten content, as shown in formula (6): （6）； Then, the input gate is used, by inputting the hidden state h from the previous time step. t-1 and the input x at the current time step t , and trainable parameters W i U i b i Calculations are performed to determine the content to be memorized, as shown in formula (7): （7）； Then, the cell state is updated, which is divided into two stages. One stage is to calculate the candidate state, as shown in formula (8), by updating the hidden state h of the previous time step. t-1 and the input x at the current time step t After linear transformation, activation is performed using the activation function tanh(•); one is to update the cell state, as shown in formula (9), by weighting and summing the candidate state, forget gate, and input gate to train the model parameters and achieve the effect of fitting the function: （8）； （9）； Among them, W c U c b c represents trainable parameters, and ⊙ represents element-wise multiplication; Finally, the output gate and the current hidden state are used as the output to provide parameters for the next LSTM update. The output gate is shown in formula (10), and the current hidden state is output by formula (11): （10）； （11）； Among them, W0, U0, and b0 are trainable parameters; For activation function: (12).

7. The UAV positioning and calibration method based on LSTM multimodal data fusion according to claim 6, characterized in that, Step 5 specifically selects the root mean square error (RMSE) to quantify the difference between the model's predicted value and the actual value. The calculation formula is as follows: （13）； Among them, y i It is the actual value. This is the predicted value, and n is the number of samples; The LSTM time series prediction model is optimized by minimizing the RMSE.

8. A computer-readable storage medium, characterized in that, It stores executable instructions that, when executed, cause the processor to perform the UAV positioning and calibration method based on LSTM multimodal data fusion according to any one of claims 1-7.

9. A computer program product, characterized in that, Includes a computer program or instructions that, when executed by a processor, implement the LSTM-based multimodal data fusion UAV positioning and calibration method according to any one of claims 1-7.

Images