GNSS / INS integrated navigation methods, devices, equipment and storage media

Abstract

This application provides a GNSS / INS integrated navigation method, apparatus, device, and storage medium, relating to the field of navigation technology. The method includes: constructing a Transformer-LSTM hybrid neural network module, taking INS data containing multiple types of information as input, and outputting predicted position and velocity increments; constructing an INS / GNSS fusion navigation module, switching inputs according to GNSS signal status, and outputting the optimal navigation solution; when GNSS is normal, the hybrid neural network module is fused using real measurements, and when the signal is abnormal, the predicted position and velocity increments are fused, ultimately outputting the optimal navigation solution; this application balances long-sequence feature extraction and short-term prediction stability, achieving high-precision position and velocity increment prediction under different GNSS interruption duration scenarios, ensuring the continuity and accuracy of navigation and positioning.

Description

Technical Field

[0001] This application relates to the field of navigation technology, and in particular to a GNSS / INS integrated navigation method, apparatus, device and storage medium. Background Technology

[0002] GNSS, with its all-weather and high-precision positioning advantages, has been widely used in autonomous driving, drone navigation, intelligent transportation, and other fields. However, in complex environments such as urban canyons, tunnels, and forests, GNSS signals are easily blocked or interfered with, leading to positioning interruptions. In such cases, INS and other auxiliary navigation systems are needed to maintain positioning continuity. However, INS suffers from cumulative errors, and its accuracy drops sharply over time. Therefore, achieving high-precision position prediction during GNSS interruptions has become a critical problem that urgently needs to be solved in the navigation field, and related research is constantly advancing around this need.

[0003] The search revealed that the most similar existing implementation schemes include the following four schemes.

[0004] Option 1: Deep integration of LSTM and robust capacitive Kalman filter (RCKF).

[0005] Step 1: Acquire time-series data from the INS, including position, velocity, attitude, accelerometer and gyroscope information, etc.

[0006] Step 2: Input the INS time series data into the LSTM neural network and use the LSTM's ability to process time series data to predict the measurement error of the inertial navigation system.

[0007] Step 3: Input the measurement error obtained from LSTM prediction into a robust capacitive Kalman filter (RCKF).

[0008] Step 4: Correct the navigation parameters of the INS through RCKF to suppress the error divergence of the navigation system and maintain navigation accuracy during GNSS interruptions.

[0009] Option 2: CNN-LSTM integrated navigation framework.

[0010] Step 1: Collect INS and GNSS data from low-cost vehicles (when GNSS is normal).

[0011] Step 2: Input the INS data into the CNN (Convolutional Neural Network) to quickly extract features from the input data;

[0012] Step 3: Input the features extracted by CNN into the LSTM network, combine them with GNSS data, and predict the covariance matrix of the Extended Kalman Filter (EKF);

[0013] Step 4: Apply the predicted covariance matrix to the EKF to achieve data fusion between INS and GNSS (when GNSS is normal). When GNSS is interrupted, navigation is maintained based on the trained model and INS data, enabling flexible navigation for low-cost vehicles.

[0014] Option 3: GNSS / INS factor graph model scheme.

[0015] Step 1: Construct a GNSS / INS factor graph model, which includes state variable nodes (such as position, velocity, attitude, etc. at different times), IMU factor nodes, GNSS factor nodes, and prior factor nodes;

[0016] Step 2: Design a weighting function based on GNSS pseudorange information;

[0017] Step 3: During the data fusion process, the weight of the GNSS factor is adaptively adjusted through the designed weight function;

[0018] Step 4: Based on probabilistic reasoning and nonlinear optimization using factor graphs, solve for the optimal navigation solution to improve the system's robustness when GNSS signals are unstable.

[0019] Option 4: INS / GNSS integrated navigation algorithm based on TransGAN.

[0020] Step 1: Construct the TransGAN architecture, with the generator using a multi-layer Transformer encoder and the discriminator using a corresponding deep learning network structure;

[0021] Step 2: Train TransGAN using INS data (position, velocity, attitude, etc.) and GNSS data (when GNSS is normal). The generator captures the global temporal features of INS data through a multi-layer Transformer encoder to improve data perception. The discriminator and generator are continuously optimized through adversarial training.

[0022] Step 3: When GNSS is interrupted, input INS data into the trained generator;

[0023] Step 4: The generator outputs an error compensation value that approximates the actual GNSS data. This error compensation value is used to correct the navigation parameters of the INS, solving the speed and position error compensation problem when GNSS is interrupted, and improving the accuracy and robustness of the integrated navigation system.

[0024] In existing solutions, while LSTM networks can capture time-series dependencies, their recurrent structure limits their ability to capture long-range feature associations. When processing long-term series data provided by INS, they struggle to effectively extract long-range dependency information, leading to decreased position and velocity prediction accuracy during prolonged GNSS outages. Transformer models, through their self-attention mechanism, can effectively extract long-series features, but they suffer from insufficient stability in short-term time-series predictions. When GNSS is interrupted for short periods and INS data fluctuates significantly in the short term, prediction results are prone to large fluctuations, failing to guarantee short-term prediction stability. The limitations of these two single models make it difficult for existing solutions to achieve high-precision and stable navigation predictions under scenarios with varying GNSS outage durations. Some existing solutions employ factor graph algorithms for data fusion, but these often fuse INS and GNSS data using fixed noise weights. When GNSS signals are interfered with and exhibit outliers, or when the accuracy of the GNSS solution changes dynamically, the fixed noise weights cannot be adjusted in time. This leads to abnormal GNSS factors significantly interfering with the fusion results, reducing data fusion accuracy, and consequently affecting the overall performance of the navigation system. Summary of the Invention

[0025] This application provides a GNSS / INS integrated navigation method, apparatus, device, and storage medium that balances long-sequence feature extraction with short-term prediction stability. It achieves high-precision position and velocity increment prediction under various GNSS outage duration scenarios, ensuring the continuity and accuracy of navigation and positioning. Simultaneously, it improves the factor graph algorithm to achieve adaptive noise covariance adjustment, adaptively adjusting the weights of abnormal GNSS factors to suppress the interference of outliers on the fusion results, thereby enhancing the accuracy and robustness of INS / GNSS data fusion. This addresses the problem that when GNSS signals are interrupted due to factors such as obstruction and interference, inertial navigation systems (INS) accumulate errors and their accuracy decreases sharply over time, leading to a loss of navigation and positioning continuity and accuracy.

[0026] Firstly, this application provides a GNSS / INS integrated navigation method, including:

[0027] A Transformer-LSTM hybrid neural network module is constructed. This module comprises a data preprocessing subunit, a position encoding subunit, a Transformer encoder subunit, and an LSTM network subunit connected sequentially. The data preprocessing subunit responds to input INS data by cleaning and normalizing the INS data, obtaining preprocessed INS data, which is then fed to the position encoding subunit. The position encoding subunit performs position encoding on the preprocessed INS data to obtain INS data containing position information, which is then fed to the Transformer encoder subunit. Based on the INS data containing position information, the Transformer encoder subunit sequentially performs multi-head self-attention processing, feedforward neural network processing, and residual connections and layer normalization to obtain feature data, which is then fed to the LSTM network subunit. The LSTM network subunit sequentially performs forget gate operation, input gate operation, memory unit update, and output gate operation on the feature data, outputting the predicted position increment and velocity increment. The INS data includes position information, velocity information, attitude information, accelerometer force, and gyroscope data.

[0028] An INS / GNSS fusion navigation module based on an improved factor graph is constructed. This module comprises, in sequence, an IMU pre-integration subunit, a factor graph construction subunit, an adaptive weight adjustment subunit, and a nonlinear optimization subunit. The input to the IMU pre-integration subunit is the IMU measurement value, and the output is the pre-integrated IMU factor. When GNSS signal reception is normal, the input to the factor graph construction subunit is the pre-integrated IMU factor, the GNSS measurement value, and initial state prior information. When GNSS signal reception is abnormal, the input to the factor graph construction subunit is the pre-integrated IMU factor, the position and velocity increments predicted by the Transformer-LSTM hybrid neural network module, and the initial state prior information. The output of the factor graph construction subunit is a complete factor graph model. The input to the adaptive weight adjustment subunit is the GNSS measurement residual, and the output is the adjusted GNSS factor weights. The input to the nonlinear optimization subunit is a weighted factor graph model, and the output is the optimal navigation solution.

[0029] When GNSS signals are being received normally, INS data is acquired and transmitted to the data preprocessing subunit of the Transformer-LSTM hybrid neural network module. GNSS measurements collected by the GNSS receiver are acquired and transmitted to the factor graph construction subunit of the INS / GNSS fusion navigation module. After processing the INS data by the Transformer-LSTM hybrid neural network module, predicted position and velocity increments are output and compared with the actual GNSS position and velocity increments. The network parameters are trained online to update the Transformer-LSTM hybrid neural network module. The IMU data is pre-integrated by the INS / GNSS fusion navigation module, and a factor graph is constructed by combining the GNSS measurements. Through adaptive weight adjustment and nonlinear optimization, the optimal navigation solution is obtained.

[0030] When GNSS signals are not received normally, INS data is acquired and transmitted to the Transformer-LSTM hybrid neural network module, which outputs the predicted position increment and velocity increment. The INS / GNSS fusion navigation module constructs a factor graph and adjusts the weights based on the IMU pre-integration factor and the predicted position increment and velocity increment, and obtains the optimal navigation solution through nonlinear optimization.

[0031] In one possible design, the location encoding subunit performs location encoding on the preprocessed INS data to obtain INS data containing location information in the following ways:

[0032] A position encoding vector is generated using sine and cosine function encoding methods. The position encoding vector is then added to the preprocessed INS data to obtain INS data containing position information.

[0033] In one possible design, the multi-head self-attention mechanism processes include:

[0034] Based on INS data containing location information, a query vector is generated for each data point through a linear transformation. Q Key vector K Sum value vector V The calculation formula is:

[0035]

[0036] Where X is the input vector, W Q , W K and W V These are query vectors Q Key vector K Sum value vector V The weight matrix;

[0037] The query vector is multiplied by all key vectors, and then the weights are normalized using the Softmax function to obtain the output of a self-attention head. The calculation formula is as follows:

[0038]

[0039] in, The output of a self-attention head; T It is the matrix transpose; d k It is the dimension of the key vector;

[0040] The outputs of all self-attention heads are concatenated and linearly transformed to obtain the output of the multi-head self-attention mechanism.

[0041] The feedforward neural network processing includes:

[0042] The output of the multi-head self-attention mechanism is input into a feedforward neural network, and independent nonlinear transformations are performed through the feedforward neural network to obtain feedforward features. The calculation formula for the independent nonlinear transformation is as follows:

[0043]

[0044] in, W 1 and W 2 represents the first and second weights. b 1 and b 2 represents the first and second bias vectors, and max is the maximum value function. y It is the output of the multi-head self-attention mechanism. FFN ( y ) is a feedforward feature;

[0045] The residual connection and layer normalization include:

[0046] After the feedforward neural network, residual connections and layer normalization are added. After the feedforward features are processed by residual connections and layer normalization, feature data is obtained.

[0047] In one possible design, the computation process of the forget gate operation is represented as follows:

[0048]

[0049] in, It is the first weight matrix. It is the first bias term. It is the Sigmoid function. f t It is the output of the forget gate. h t-1These are the predicted position and velocity increments from the previous moment. x t It is the input information at the current moment;

[0050] The calculation process of the input gate operation is expressed as follows:

[0051]

[0052]

[0053] in, It is the second weight matrix. It is the second bias term. i t It is the core output of the input gate, used to determine which new information will be updated in the cell state (long-term memory). It is the third weight matrix. It is the third bias term, and tanh is the activation function. This represents a "candidate new memory" jointly generated by the current input and the previous state;

[0054] The calculation process for updating the memory unit is represented as follows:

[0055]

[0056] in, C t It is the current state of the memory unit. C t-1 It is the state of the memory unit in the previous moment. It is element-wise multiplication;

[0057] The calculation process for the output gate operation is expressed as follows:

[0058]

[0059]

[0060] in, It is the fourth weight matrix. It is the fourth bias term. h t These are the predicted position and velocity increments at the current moment. It is the output mask.

[0061] In one possible design, the IMU pre-integration subunit calculates the IMU pre-integration factor based on the nonlinear differential equations of the state changes of the inertial navigation system using the following formula:

[0062]

[0063] in, For IMU pre-integration factor, x i Let i be the system state variable at time i. x i-1 Let i be the system state variable at time i-1. d Let cost function be The state transition function of the IMU. Let be the inertial sensor bias variable at time i-1. This is the IMU measurement at time i-1.

[0064] In one possible design, the factor graph construction subunit receives IMU pre-integration factors, GNSS measurements, and initial state prior information, and constructs a factor graph model including state variable nodes, IMU pre-integration factor nodes, GNSS factor nodes, and prior factor nodes; wherein, the formula for calculating the GNSS measurements is:

[0065]

[0066] in, For GNSS observations, For observation models, For measuring noise;

[0067] The calculation formula for the GNSS factor node is as follows:

[0068]

[0069] in, For GNSS factor nodes;

[0070] The calculation formula for the prior factor node is as follows:

[0071]

[0072] in, Let be the prior factor, and exp be the exponential function. x For system state variables, This is the mean of the prior information (the highest prior estimate). It is the square of the Mahalanobis distance.

[0073] In one possible design, the adaptive weight adjustment subunit adjusts the GNSS factor weights based on the calculated GNSS measurement residuals using a weighting function to obtain the adjusted GNSS factor weights; wherein, the formula for calculating the GNSS measurement residuals is:

[0074]

[0075] in, This is an estimated value. For measured values, For GNSS measurement residuals;

[0076] The weighting function is expressed as follows:

[0077]

[0078] in, The adjusted GNSS factor weights, This is the residual threshold.

[0079] Secondly, this application provides a GNSS / INS integrated navigation device, the device comprising:

[0080] The first construction module is configured to build a Transformer-LSTM hybrid neural network module. This module includes a data preprocessing subunit, a position encoding subunit, a Transformer encoder subunit, and an LSTM network subunit connected sequentially. The data preprocessing subunit responds to the input INS data, performs data cleaning and normalization on the INS data, and feeds the preprocessed INS data to the position encoding subunit. The position encoding subunit performs position encoding on the preprocessed INS data to obtain INS data containing position information, and feeds it to the Transformer encoder subunit. Based on the INS data containing position information, the Transformer encoder subunit sequentially performs multi-head self-attention processing, feedforward neural network processing, and residual connections and layer normalization to obtain feature data, which is then fed to the LSTM network subunit. The LSTM network subunit sequentially performs forget gate operation, input gate operation, memory unit update, and output gate operation on the feature data, outputting the predicted position increment and velocity increment. The INS data includes position information, velocity information, attitude information, accelerometer force, and gyroscope data.

[0081] The second construction module is configured to construct an INS / GNSS fusion navigation module based on an improved factor graph. This INS / GNSS fusion navigation module includes, in sequence, an IMU pre-integration subunit, a factor graph construction subunit, an adaptive weight adjustment subunit, and a nonlinear optimization subunit. The input to the IMU pre-integration subunit is the IMU measurement value, and the output is the pre-integrated IMU factor. When the GNSS signal is normally received, the input to the factor graph construction subunit is the pre-integrated IMU factor, the GNSS measurement value, and initial state prior information. When the GNSS signal is not normally received, the input to the factor graph construction subunit is the pre-integrated IMU factor, the position and velocity increments predicted by the Transformer-LSTM hybrid neural network module, and the initial state prior information. The output of the factor graph construction subunit is a complete factor graph model. The input to the adaptive weight adjustment subunit is the GNSS measurement residual, and the output is the adjusted GNSS factor weights. The input to the nonlinear optimization subunit is a weighted factor graph model, and the output is the optimal navigation solution.

[0082] The first collaborative module is configured to acquire INS data and transmit it to the data preprocessing subunit of the Transformer-LSTM hybrid neural network module when GNSS signals are being received normally; acquire GNSS measurement values ​​collected by the GNSS receiver and transmit them to the factor graph construction subunit of the INS / GNSS fusion navigation module; process the INS data through the Transformer-LSTM hybrid neural network module and output the predicted position and velocity increments, compare them with the actual GNSS position and velocity increments, train the network parameters online, and update the Transformer-LSTM hybrid neural network module; and pre-integrate the IMU data through the INS / GNSS fusion navigation module, construct a factor graph by combining it with the GNSS measurement values, and obtain the optimal navigation solution through adaptive weight adjustment and nonlinear optimization.

[0083] The second collaborative working module is configured to acquire INS data and transmit it to the Transformer-LSTM hybrid neural network module when GNSS signals are not received normally, and output the predicted position increment and velocity increment. The INS / GNSS fusion navigation module constructs a factor graph and adjusts the weights based on the IMU pre-integration factor and the predicted position increment and velocity increment, and obtains the optimal navigation solution through nonlinear optimization.

[0084] Thirdly, embodiments of this application provide an electronic device, including: at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, causing the at least one processor to perform the GNSS / INS integrated navigation method as described in the first aspect and various possible designs of the first aspect.

[0085] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions. When a processor executes the computer-executable instructions, it implements the GNSS / INS integrated navigation method described in the first aspect and various possible designs of the first aspect.

[0086] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the GNSS / INS integrated navigation method as described in the first aspect and various possible designs of the first aspect.

[0087] The GNSS / INS integrated navigation method, apparatus, equipment, and storage medium provided in this application have at least the following beneficial effects:

[0088] 1) This application employs a Transformer-LSTM hybrid neural network. The Transformer encoder, through a multi-head self-attention mechanism, can effectively extract the spatiotemporal joint features and long-range dependencies of long-sequence INS data, solving the problem of insufficient long-range feature capture capability of LSTM. The LSTM network, through a gating structure, can accurately capture short-term temporal dependencies, solving the problem of poor short-term prediction stability of Transformer. The combination of the two enables high-precision position and velocity increment prediction under different interruption durations during long-term GNSS interruptions.

[0089] 2) This application improves the factor graph algorithm by introducing an adaptive weight adjustment mechanism based on GNSS measurement residuals. When outliers appear in the GNSS signal, the reliability is assessed by calculating the measurement residuals, and the GNSS factor weights are dynamically adjusted. The weights of factors with large residuals (low reliability) are reduced, while the weights of factors with small residuals (high reliability) are increased, effectively suppressing outlier interference and improving data fusion accuracy. Attached Figure Description

[0090] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0091] Figure 1 A flowchart illustrating a GNSS / INS integrated navigation method provided in this application embodiment;

[0092] Figure 2 This is a structural diagram of the Transformer encoder subunit provided in an embodiment of this application;

[0093] Figure 3 This is a structural diagram of an LSTM network subunit provided in an embodiment of this application;

[0094] Figure 4 A data processing flowchart of the Transformer-LSTM hybrid neural network module provided in the embodiments of this application;

[0095] Figure 5 A flowchart of the Transformer-LSTM hybrid neural network module provided in this application embodiment;

[0096] Figure 6 This is a structural diagram of the GNSS / INS integrated navigation factor graph provided in the embodiments of this application;

[0097] Figure 7 An improved factor graph model structure diagram provided in the embodiments of this application;

[0098] Figure 8 Another flowchart of a GNSS / INS integrated navigation method provided in this application embodiment;

[0099] Figure 9 This is a structural diagram of the GNSS / INS integrated navigation device provided in the embodiments of this application.

[0100] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0101] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0102] The collection, storage, use, processing, transmission, provision, and disclosure of financial data or user data involved in the technical solution of this application all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0103] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0104] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0105] This application provides a GNSS / INS integrated navigation method, such as... Figure 1 As shown, the GNSS / INS integrated navigation method includes the following steps S10 to S40.

[0106] S10: Construct the Transformer-LSTM hybrid neural network module.

[0107] In this embodiment, the Transformer-LSTM hybrid neural network module includes a data preprocessing subunit, a position encoding subunit, a Transformer encoder subunit, and an LSTM network subunit, all connected sequentially. The input to the data preprocessing subunit is the 15-dimensional data provided by the INS, and the output of the LSTM network subunit is the predicted position and velocity increments. The Transformer encoder subunit structure is as follows: Figure 2 As shown, it consists of n identical encoders stacked together. Each encoder contains a multi-head self-attention sublayer and a feed-forward neural network sublayer. Each sublayer is followed by a residual connection and a layer normalization operation. Figure 2 In this context, the input token segmentation is processed by the data preprocessing subunit to obtain preprocessed INS data, which includes position information, velocity information, attitude information, accelerometer specific force, and gyroscope data. The position encoding process is executed by the position encoding subunit.

[0108] The structure of the LSTM network subunit is as follows: Figure 3As shown, it includes three gating structures: a forget gate, an input gate, and an output gate, as well as a memory unit. The gating structures control the forgetting, updating, and output of information, thereby enabling the processing of time-series data.

[0109] The data processing flow of the Transformer-LSTM hybrid neural network module is as follows: Figure 4 As shown, the preprocessed INS data is used as the INS input data. After being processed by position encoding, multi-head self-attention mechanism, feedforward neural network, residual connection and layer normalization, the resulting feature data is activated by the activation function. Combined with the hidden state and the unit state of the previous time step, it is processed by LSTM neural network, and then processed by activation function and fully connected layer to obtain the output value. The output value is the predicted position increment and velocity increment.

[0110] like Figure 5 As shown, the Transformer-LSTM hybrid neural network module includes the following steps S110-S140 when it is working.

[0111] S110: Data preprocessing.

[0112] The data preprocessing subunit receives 15-dimensional data from the INS, including position, velocity, attitude, accelerometer force, and gyroscope data. It cleans the 15-dimensional data to remove outliers and normalizes the data to scale it to a uniform range, ensuring data consistency and validity, and providing high-quality data for subsequent processing. Through these preprocessing steps, preprocessed INS data is obtained.

[0113] S120: Location code.

[0114] The position encoding subunit receives the preprocessed INS data. Since the Transformer encoder lacks built-in sequence processing capabilities, position encoding is required to provide sequence position information for the data. A sine and cosine function encoding method is used to generate a position encoding vector, which is then added to the input INS data vector (i.e., the preprocessed INS data) to obtain INS data containing position information, which is then transmitted to the Transformer encoder subunit.

[0115] S130: Transformer encoder processing, including the following steps S131-S133.

[0116] S131: Multi-head self-attention mechanism processing.

[0117] The Transformer encoder subunit receives INS data containing location information and generates a vector of query (Q), key (K), and value (V) for each data point through a linear transformation, using the following formula:

[0118]

[0119] Where X is the input vector, W Q , W K and W V These are query vectors Q Key vector K Sum value vector V The weight matrix.

[0120] Perform a dot product operation on the query vector and all key vectors, then normalize the weights using the Softmax function. The formula is as follows:

[0121]

[0122] in, The output of a self-attention head; T It is the matrix transpose; d k It is the dimension of the key vector.

[0123] The outputs of all heads are concatenated and linearly transformed to obtain the final output of the multi-head self-attention mechanism.

[0124] S132: Feedforward neural network processing.

[0125] The output of the multi-head self-attention mechanism is input into a feedforward neural network. This network performs an independent nonlinear transformation on the output at each position, consisting of two linear transformations and a ReLU activation function, as shown in the following formula:

[0126]

[0127] in, W 1 and W 2 represents the first and second weights. b 1 and b 2 represents the first and second bias vectors, and max is the maximum value function. y It is the output of the multi-head self-attention mechanism. FFN ( y ) is a feedforward feature.

[0128] S133: Residual connectivity and layer normalization.

[0129] After the multi-head self-attention mechanism sub-layer and the feedforward neural network sub-layer, residual connections (adding the sub-layer input and output to alleviate the gradient vanishing problem in deep networks) and layer normalization are added respectively to ensure that the output of each layer remains stable during training. The processed feature data is then passed to the LSTM network sub-unit. The formula for layer normalization is:

[0130]

[0131] in, , These are the mean and variance of the input, respectively. z It is the input vector. This is to avoid positive constants with a denominator of 0. These are scaling parameters that adjust the normalized features by scaling them. It is an offset parameter that adjusts the offset of the normalized features. LN ( z ) is the output of layer normalization.

[0132] S140: LSTM network processing, including the following steps S141-S144.

[0133] S141: Forgot Gate Operation.

[0134] The LSTM network sub-unit receives the feature data output by the Transformer encoder. The forget gate input is the feature data at the current time step and the hidden state at the previous time step. The Sigmoid activation function determines the information to be discarded from the memory unit. The closer the output value is to 1, the more information is retained. The closer it is to 0, the more information is forgotten.

[0135] The calculation process for the input gate operation is represented as follows:

[0136]

[0137]

[0138] in, It is the second weight matrix. It is the second bias term. i t It is the core output of the input gate, used to determine which new information will be updated in the cell state (long-term memory). It is the third weight matrix. It is the third bias term, and tanh is the activation function. It represents a "candidate new memory" jointly generated by the current input and the previous state.

[0139] S142: Input gate operation.

[0140] The input gate generates an "update mask" using the Sigmoid function to control information updates; then it generates candidate update values ​​(ranging from -1 to 1) using the tanh function.

[0141] The calculation process for the input gate operation is represented as follows:

[0142]

[0143]

[0144] in, It is the second weight matrix. It is the second bias term. i t It is the core output of the input gate, used to determine which new information will be updated in the cell state (long-term memory). It is the third weight matrix. It is the third bias term, and tanh is the activation function. It represents a "candidate new memory" jointly generated by the current input and the previous state.

[0145] S143: Memory unit update.

[0146] By combining the forget gate and the input gate to update the state of the memory unit, the information of the memory unit can be updated.

[0147] The computational process for updating memory cells is represented as follows:

[0148]

[0149] in, C t It is the current state of the memory unit. C t-1 It is the state of the memory unit in the previous moment. It is element-wise multiplication.

[0150] S144: Output gate operation.

[0151] The output gate generates an output mask using the Sigmoid function, applies the tanh function to compress the memory cell state to -1 to 1, and then multiplies it with the output mask to obtain the current hidden state. Finally, it outputs the predicted position increment and velocity increment.

[0152] The calculation process for the output gate operation is expressed as follows:

[0153]

[0154]

[0155] in, It is the fourth weight matrix. It is the fourth bias term. h t These are the predicted position and velocity increments at the current moment. It is the output mask.

[0156] S20: Construct an INS / GNSS fusion navigation module based on an improved factor graph.

[0157] In this embodiment, the INS / GNSS fusion navigation module includes an IMU pre-integration subunit, a factor graph construction subunit, an adaptive weight adjustment subunit, and a nonlinear optimization subunit connected in sequence. The input of the IMU pre-integration subunit is the IMU measurement value, and the output is the pre-integrated IMU factor. When the GNSS signal is normally received, the input of the factor graph construction subunit is the pre-integrated IMU factor, the GNSS measurement value, and the initial state prior information. When the GNSS signal is not normally received, the input of the factor graph construction subunit is the pre-integrated IMU factor, the position increment and velocity increment predicted by the Transformer-LSTM hybrid neural network module, and the initial state prior information. The output of the factor graph construction subunit is the complete factor graph model. The input of the adaptive weight adjustment subunit is the GNSS measurement residual, and the output is the adjusted GNSS factor weight. The input of the nonlinear optimization subunit is the weighted factor graph model, and the output is the optimal navigation solution.

[0158] The structure of the GNSS / INS integrated navigation factor graph is as follows: Figure 6 As shown, the improved factor graph model structure is as follows: Figure 7 As shown, Figure 6 and Figure 7 middle, To constrain the prior factors of the initial system state, and These are the IMU pre-integration factors at time 1 and time 2, respectively. x 0、 x 1. x 2 and x n Let be the system state variables at time 0, time 1, time 2, and time n, respectively. To constrain the prior factors of the initial sensor bias, , , and The inertial sensor bias variables at time 0, time 1, time 2, and time n are respectively. , and These are the GNSS factors at time 1, time 2, and time n, respectively. , and These are the weighting factors for GNSS at time 1, time 2, and time n, respectively.

[0159] The IMU pre-integration subunit receives high-frequency acceleration and angular velocity measurements from the IMU output. To avoid redundant information increasing the computational burden, multiple IMU measurements are integrated into a single IMU pre-integration factor node within a certain sampling interval. The IMU pre-integration factor is calculated based on the nonlinear differential equation of the inertial navigation system's state change, using the following formula:

[0160]

[0161] in, For IMU pre-integration factor, x i Let i be the system state variable at time i. x i-1 Let i be the system state variable at time i-1. d Let cost function be The state transition function of the IMU. Let be the inertial sensor bias variable at time i-1. This represents the IMU measurement at time i-1.

[0162] The IMU pre-integration subunit passes the pre-integrated IMU factors to the factor graph construction subunit.

[0163] The nonlinear differential equation for the state change of an inertial navigation system is expressed as:

[0164]

[0165] in, Let t be the pose state at time t. For inertial sensor bias variables, For acceleration and angular velocity measured by the IMU, It is the derivative of the system state.

[0166] In one embodiment, the IMU pre-integrating subunit simultaneously considers the IMU nonlinear error model, and the deviation factor node formula is as follows:

[0167]

[0168] in, f bias For IMU bias factor, g imu For IMU bias prediction function, and are the IMU bias variables at time i and time i-1.

[0169] When GNSS is normal, the factor graph construction sub-unit receives IMU pre-integration factors, GNSS measurements and initial state prior information, and constructs a factor graph model containing state variable nodes, IMU pre-integration factor nodes, GNSS factor nodes and prior factor nodes.

[0170] The formula for calculating GNSS measurement values ​​is:

[0171]

[0172] in, For GNSS observations, For observation models, For measuring noise;

[0173] The formula for calculating GNSS factor nodes is:

[0174]

[0175] in, For GNSS factor nodes;

[0176] The formula for calculating the prior factor node is:

[0177]

[0178] in, Let be the prior factor, and exp be the exponential function. x For system state variables, This is the mean of the prior information (the highest prior estimate). It is the square of the Mahalanobis distance.

[0179] The adaptive weight adjustment sub-unit calculates GNSS measurement residuals and evaluates the reliability of GNSS measurements based on the residual magnitude. A weighting function is used to adjust the GNSS factor weights: smaller residuals result in larger weights, and larger residuals result in smaller weights, suppressing outlier interference. The adjusted weights are then passed to the factor graph construction sub-unit to update the factor graph model.

[0180] The formula for calculating GNSS measurement residuals is:

[0181]

[0182] in, This is an estimated value. For measured values, For GNSS measurement residuals.

[0183] The weighting function is expressed as:

[0184]

[0185] in, The adjusted GNSS factor weights, This is the residual threshold.

[0186] S30: When GNSS signals are being received normally, INS data is acquired and transmitted to the data preprocessing subunit of the Transformer-LSTM hybrid neural network module. GNSS measurements collected by the GNSS receiver are acquired and transmitted to the factor graph construction subunit of the INS / GNSS fusion navigation module. After processing the INS data by the Transformer-LSTM hybrid neural network module, the predicted position and velocity increments are output and compared with the actual GNSS position and velocity increments. The network parameters are trained online to update the Transformer-LSTM hybrid neural network module. The IMU data is pre-integrated by the INS / GNSS fusion navigation module, and a factor graph is constructed by combining the GNSS measurements. The optimal navigation solution is obtained through adaptive weight adjustment and nonlinear optimization.

[0187] In this embodiment, during the normal GNSS phase, the INS collects 15-dimensional data (INS data) in real time and transmits it to the data preprocessing subunit of the Transformer-LSTM hybrid neural network module. Simultaneously, the GNSS receiver collects GNSS measurements and transmits them to the factor graph construction subunit of the improved factor graph fusion navigation module. After processing the INS data, the Transformer-LSTM hybrid neural network module outputs predicted position and velocity increments, which are compared with the actual GNSS position and velocity increments to train network parameters online and optimize model performance. The INS / GNSS fusion navigation module based on the improved factor graph pre-integrates the IMU data, constructs a factor graph using GNSS measurements, and obtains and outputs the optimal navigation solution through adaptive weight adjustment and nonlinear optimization.

[0188] S40: When GNSS signals are not received normally, acquire INS data and transmit it to the Transformer-LSTM hybrid neural network module, outputting the predicted position increment and velocity increment. The INS / GNSS fusion navigation module constructs a factor graph and adjusts the weights based on the IMU pre-integration factor and the predicted position increment and velocity increment, and obtains the optimal navigation solution through nonlinear optimization.

[0189] In this embodiment, during the GNSS interruption phase, the INS continues to collect 15-dimensional data (INS data) and transmits it to the pre-trained Transformer-LSTM hybrid neural network module. The module outputs accurate predicted position and velocity increments (pseudo-GNSS measurement information). The improved factor graph fusion navigation module receives the IMU pre-integration factor and pseudo-GNSS measurement information, constructs a factor graph and adjusts the weights, and obtains the optimal navigation solution through nonlinear optimization, ensuring uninterrupted navigation and maintaining high-precision positioning.

[0190] In another embodiment, such as Figure 8 The diagram shown is another flowchart of a GNSS / INS integrated navigation method provided in this application embodiment. This method, based on acquired 15-dimensional INS data, preprocesses the data and inputs the preprocessed data into a Transformer encoder to extract spatiotemporal joint features (corresponding to the aforementioned feature data). Then, it determines whether the GNSS signal is normal by whether a GNSS signal is received; if a GNSS signal is received, the GNSS signal is normal; otherwise, it is abnormal. When the GNSS signal is normal, an improved factor graph algorithm is used to fuse GNSS observations and INS features to achieve navigation output. When the GNSS signal is abnormal, an LSTM network is used to predict the position increment as a GNSS observation, and the improved factor graph algorithm is used to fuse GNSS observations and INS features to achieve navigation output.

[0191] In summary, this GNSS / INS integrated navigation method utilizes the multi-head self-attention mechanism of the Transformer encoder to extract spatiotemporal joint features and long-range dependencies from the 15-dimensional data (including position, velocity, attitude, accelerometer, and gyroscope information) provided by INS. By capturing short-term temporal dependencies through the structure of an LSTM network, it effectively improves the accuracy of position and velocity increment predictions during GNSS outages, providing reliable data support for subsequent navigation corrections. Furthermore, the improved factor graph algorithm achieves adaptive noise covariance adjustment to enhance fusion robustness, adaptively adjusting the weights of anomalous GNSS factors to suppress the interference of outliers on the fusion results, thereby improving the accuracy and robustness of INS / GNSS data fusion.

[0192] This method effectively improves the stability and accuracy of position and velocity prediction in GNSS interruption scenarios, effectively achieving high-precision continuous navigation. To prove that this method can effectively achieve the above effects, this embodiment verifies it through simulation experiments. In the simulation experiment, the sampling frequency of the vehicle test data is 5Hz, the simulation time is 500s, the Transformer-LSTM model is trained for the first 400s, and the GNSS interruption is simulated for 400-500s. The experiment compares three methods: LSTM + ordinary factor map, LSTM + improved factor map, and the present invention (Transformer-LSTM + improved factor map). The results are shown in Table 1 (position error of different methods) and Table 2 (velocity error of different methods). The method proposed in this application is superior to the other two methods in terms of RMSE (root mean square error) and MAE (mean absolute error) on the X, Y, and Z axes, and the error curve is flatter and fluctuates less during GNSS interruption, proving the superiority of the proposed solution.

[0193] It should be noted that in the experiment, the LSTM neural network was set to 3 hidden layers with 128 hidden units; the Transformer encoder was set to 4 layers with 8 attention heads, a feedforward dimension of 512, and a sequence length of 10 time steps. These parameter settings ensured both prediction accuracy and computational efficiency, and can serve as a reference for adjusting model parameters in subsequent practical applications.

[0194] Table 1 Comparison of position errors of the three methods

[0195]

[0196] Table 2 Comparison of speed errors among the three methods

[0197]

[0198] It should be noted that although the method described in the above embodiments is mainly aimed at INS / GNSS integrated navigation in GNSS interruption scenarios, the above method can also be extended to multi-sensor fusion navigation scenarios (such as combining visual sensors and LiDAR). The predictive ideas of the Transformer-LSTM hybrid neural network and the adaptive fusion ideas of improved factor graphs are applied to multi-sensor data processing to further improve navigation reliability in extreme environments. For example, GNSS measurements can be equivalently replaced by data collected by visual sensors or LiDAR. When the visual sensors or LiDAR do not provide data in a timely manner or the provided data cannot be applied to navigation, it corresponds to the GNSS interruption stage in the above method. When the visual sensors or LiDAR have provided data and the data can be applied to navigation, it corresponds to the GNSS normal stage in the above method.

[0199] This application also provides a GNSS / INS integrated navigation device, such as... Figure 9 As shown, the GNSS / INS integrated navigation device includes:

[0200] The first construction module 901 is configured to construct a Transformer-LSTM hybrid neural network module. The Transformer-LSTM hybrid neural network module includes a data preprocessing subunit, a position encoding subunit, a Transformer encoder subunit, and an LSTM network subunit connected sequentially. The data preprocessing subunit responds to the input INS data and performs data cleaning and normalization on the INS data to obtain preprocessed INS data, which is then fed to the position encoding subunit. The position encoding subunit performs position encoding on the preprocessed INS data to obtain data containing the position... The INS data containing position information is fed to the Transformer encoder subunit. Based on the INS data containing position information, the Transformer encoder subunit sequentially performs multi-head self-attention mechanism processing, feedforward neural network processing, and residual connection and layer normalization to obtain feature data, which is then fed to the LSTM network subunit. The LSTM network subunit sequentially performs forget gate operation, input gate operation, memory unit update, and output gate operation on the feature data to output the predicted position increment and velocity increment. The INS data includes position information, velocity information, attitude information, accelerometer force, and gyroscope data.

[0201] The second construction module 902 is configured to construct an INS / GNSS fusion navigation module based on an improved factor graph. The INS / GNSS fusion navigation module includes, in sequence, an IMU pre-integration subunit, a factor graph construction subunit, an adaptive weight adjustment subunit, and a nonlinear optimization subunit. The input of the IMU pre-integration subunit is the IMU measurement value, and the output is the pre-integrated IMU factor. When the GNSS signal is normally received, the input of the factor graph construction subunit is the pre-integrated IMU factor, the GNSS measurement value, and initial state prior information. When the GNSS signal is not normally received, the input of the factor graph construction subunit is the pre-integrated IMU factor, the position and velocity increments predicted by the Transformer-LSTM hybrid neural network module, and the initial state prior information. The output of the factor graph construction subunit is a complete factor graph model. The input of the adaptive weight adjustment subunit is the GNSS measurement residual, and the output is the adjusted GNSS factor weights. The input of the nonlinear optimization subunit is a weighted factor graph model, and the output is the optimal navigation solution.

[0202] The first collaborative module 903 is configured to acquire INS data and transmit it to the data preprocessing subunit of the Transformer-LSTM hybrid neural network module when GNSS signals are being received normally; acquire GNSS measurement values ​​collected by the GNSS receiver and transmit them to the factor graph construction subunit of the INS / GNSS fusion navigation module; process the INS data through the Transformer-LSTM hybrid neural network module and output the predicted position and velocity increments, compare them with the actual GNSS position and velocity increments, train the network parameters online, and update the Transformer-LSTM hybrid neural network module; pre-integrate the IMU data through the INS / GNSS fusion navigation module, construct a factor graph by combining it with the GNSS measurement values, and obtain the optimal navigation solution through adaptive weight adjustment and nonlinear optimization.

[0203] The second collaborative working module 904 is configured to acquire INS data and transmit it to the Transformer-LSTM hybrid neural network module when GNSS signals are not received normally, and output the predicted position increment and velocity increment. The INS / GNSS fusion navigation module constructs a factor graph and adjusts the weights based on the IMU pre-integration factor and the predicted position increment and velocity increment, and obtains the optimal navigation solution through nonlinear optimization.

[0204] This application provides an electronic device. The electronic device may include a processor and a memory, wherein the processor and the memory can communicate; exemplarily, the processor and the memory communicate via a communication bus.

[0205] The processor executes computer execution instructions stored in memory, causing the processor to perform the scheme in the above embodiments. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0206] The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0207] The electronic device provided in this application embodiment can be the terminal device described in the above embodiments.

[0208] This application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a computer, the computer performs the technical solution of the GNSS / INS integrated navigation method described in the above embodiments.

[0209] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and when the at least one processor executes the computer program, it can implement the technical solution of the GNSS / INS integrated navigation method in the above embodiments.

[0210] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0211] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0212] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit composed of the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0213] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0214] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0215] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk or optical disc, etc.

[0216] Buses can be Industry Standard Architecture (ISA) buses, Peripheral Component Interconnect (PCI) buses, or Extended Industry Standard Architecture (EISA) buses, etc. Buses can be categorized into address buses, data buses, control buses, etc.

[0217] The aforementioned storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0218] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. The processor and storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components in an electronic control unit or main control device.

[0219] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0220] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A GNSS / INS integrated navigation method, characterized in that, The method comprises: The Transformer-LSTM hybrid neural network module comprises a data preprocessing subunit, a position coding subunit, a Transformer encoder subunit and an LSTM network subunit connected in sequence, the data preprocessing subunit is responsive to input INS data, and performs data cleaning and normalization on the INS data to obtain preprocessed INS data and feed the preprocessed INS data to the position coding subunit, the position coding subunit performs position coding on the preprocessed INS data to obtain INS data containing position information and feed the INS data containing position information to the Transformer encoder subunit, the Transformer encoder subunit performs multi-head self-attention mechanism processing, feedforward neural network processing, residual connection and layer normalization based on the INS data containing position information in sequence to obtain feature data and feed the feature data to the LSTM network subunit, and the LSTM network subunit performs forget gate operation, input gate operation, memory cell update and output gate operation on the feature data in sequence to output predicted position increment and velocity increment; the INS data comprises position information, velocity information, attitude information, accelerometer specific force and gyroscope data; An INS / GNSS fusion navigation module based on an improved factor graph is constructed, the INS / GNSS fusion navigation module comprises an IMU pre-integration subunit, a factor graph construction subunit, an adaptive weight adjustment subunit and a nonlinear optimization subunit connected in sequence, the input of the IMU pre-integration subunit is IMU measurement values, and the output is pre-integrated IMU factors; when GNSS signals are normally received, the input of the factor graph construction subunit is pre-integrated IMU factors, GNSS measurement values and initial state prior information, when GNSS signals are not normally received, the input of the factor graph construction subunit is pre-integrated IMU factors, position increment and velocity increment predicted by the Transformer-LSTM hybrid neural network module and initial state prior information, and the output of the factor graph construction subunit is a complete factor graph model; the input of the adaptive weight adjustment subunit is GNSS measurement residual, and the output is adjusted GNSS factor weight; the input of the nonlinear optimization subunit is the factor graph model with weight, and the output is an optimal navigation solution; When the GNSS signal is normally received, the INS data is acquired and transmitted to the data preprocessing subunit of the Transformer-LSTM hybrid neural network module, the GNSS measurement value collected by the GNSS receiver is acquired and transmitted to the factor graph construction subunit of the INS / GNSS fusion navigation module, after the INS data is processed by the Transformer-LSTM hybrid neural network module, the predicted position increment and velocity increment are output, and the network parameters are trained online by comparison with the real GNSS position increment and velocity increment, so as to update the Transformer-LSTM hybrid neural network module; the IMU data is pre-integrated by the INS / GNSS fusion navigation module, and the factor graph is constructed in combination with the GNSS measurement value, and the optimal navigation solution is obtained through adaptive weight adjustment and nonlinear optimization; When the GNSS signal is not normally received, the INS data is acquired and transmitted to the Transformer-LSTM hybrid neural network module, and the predicted position increment and velocity increment are output, the INS / GNSS fusion navigation module constructs a factor graph and adjusts the weight according to the IMU pre-integrated factor and the predicted position increment and velocity increment, and obtains the optimal navigation solution through nonlinear optimization.

2. The GNSS / INS integrated navigation method of claim 1, wherein, The position encoding subunit encodes the preprocessed INS data to obtain INS data containing position information in the following manner: A sine and cosine function encoding method is adopted to generate a position encoding vector, and the position encoding vector is added to the preprocessed INS data to obtain INS data containing position information.

3. The GNSS / INS integrated navigation method of claim 1, wherein, The multi-head self-attention mechanism processing includes: Based on the INS data containing position information, a query vector is generated for each data point through a linear transformation Q , a key vector K , and a value vector V , the calculation formula is: where X is an input vector, W Q , W K and W V are weight matrices of query vector Q , key vector K and value vector V , respectively. The dot product operation is performed on the query vector and all key vectors, and the weight is normalized through the Softmax function to obtain the output of one self-attention head, and the calculation formula is: wherein, is the output of a self-attention head; T is the matrix transpose; d k is the dimension of the key vector; The outputs of all self-attention heads are spliced, and the output of the multi-head self-attention mechanism is obtained through linear transformation; The feedforward neural network processing includes: The output of the multi-head self-attention mechanism is input into the feedforward neural network, and independent nonlinear transformation is performed on the feedforward neural network to obtain feedforward features, and the calculation formula of the independent nonlinear transformation is: wherein, W 1 and W 2 are a first weight and a second weight, b 1 and b 2 are a first bias vector and a second bias vector, max is a max function, y is an output of a multi-head self-attention mechanism, FFN y is a feed-forward feature;​ The residual connection and layer normalization include: After the feedforward neural network, the residual connection and layer normalization are added, and after the residual connection and layer normalization processing of the feedforward features, the feature data is obtained.

4. The GNSS / INS integrated navigation method of claim 1, wherein, The calculation process of the forgetting gate operation is represented as: wherein, is a first weight matrix, is a first bias term, is a Sigmoid function, f t is an output of a forget gate, h t-1 is a predicted position increment and velocity increment at a previous time, x t is input information at a current time; The calculation process of the input gate operation is represented as: wherein, is a second weight matrix, is a second bias term, i t is the core output of the input gate, which determines which new information will be updated into the cell state, is a third weight matrix, is a third bias term, and tanh is an activation function, represents the candidate new memory jointly generated by the current input and the previous state; The calculation process of the memory cell update is represented as: wherein, C t is the current time memory cell state, C t-1 is the previous time memory cell state, is an element-wise multiplication; The calculation process of the output gate operation is represented as: wherein, is a fourth weight matrix, is a fourth bias term, h t is a predicted position increment and velocity increment at a current time instant, is an output mask.

5. The GNSS / INS integrated navigation method of claim 1, wherein, The IMU pre-integrated subunit calculates the IMU pre-integrated factor according to the nonlinear differential equation of the inertial navigation system state change through the following formula: wherein, is an IMU pre-integration factor, x i is the system state variable at time i, x i-1 is the system state variable at time i-1, d is a cost function, is a state transition function for the IMU, is the inertial sensor bias variable at time i-1, is the IMU measurement at time i-1.

6. The GNSS / INS integrated navigation method of claim 5, wherein, The factor graph construction subunit receives the IMU pre-integrated factor, the GNSS measurement value and the initial state prior information, and constructs a factor graph model containing state variable nodes, IMU pre-integrated factor nodes, GNSS factor nodes and prior factor nodes; wherein, the calculation formula of the GNSS measurement value is: wherein, is a GNSS observation, is an observation model, is a measurement noise; The calculation formula of the GNSS factor node is: wherein, GNSS factor node; The calculation formula of the prior factor node is: wherein, is a prior factor, exp is the exponential function, x is a system state variable, is a mean of the prior information, is the square of the Mahalanobis distance.

7. The GNSS / INS integrated navigation method of claim 6, wherein, The adaptive weight adjustment subunit adjusts the GNSS factor weight by using a weight function based on the calculated GNSS measurement residual, to obtain an adjusted GNSS factor weight; wherein the calculation formula of the GNSS measurement residual is: wherein, is an estimated value, is a measured value, is a GNSS measurement residual; The weight function is represented as: wherein, is the adjusted GNSS factor weight, is a residual threshold.

8. A GNSS / INS integrated navigation device, characterized by, The device comprises: The first construction module is configured to construct a Transformer-LSTM hybrid neural network module; wherein the Transformer-LSTM hybrid neural network module comprises a data preprocessing subunit, a position encoding subunit, a Transformer encoder subunit and an LSTM network subunit connected in sequence, the data preprocessing subunit is responsive to input INS data, and performs data cleaning and normalization on the INS data to obtain preprocessed INS data and feeds it to the position encoding subunit, the position encoding subunit performs position encoding on the preprocessed INS data to obtain INS data containing position information and feeds it to the Transformer encoder subunit, the Transformer encoder subunit performs multi-head self-attention mechanism processing, feedforward neural network processing, residual connection and layer normalization in sequence based on the INS data containing position information to obtain feature data and feeds it to the LSTM network subunit, and the LSTM network subunit performs forget gate operation, input gate operation, memory cell update and output gate operation in sequence on the feature data to output predicted position increment and velocity increment; the INS data comprises position information, velocity information, attitude information, accelerometer specific force and gyroscope data; The second construction module is configured to construct an INS / GNSS fusion navigation module based on an improved factor graph, the INS / GNSS fusion navigation module comprises an IMU pre-integration subunit, a factor graph construction subunit, an adaptive weight adjustment subunit and a nonlinear optimization subunit connected in sequence, the input of the IMU pre-integration subunit is IMU measurement value, and the output is pre-integrated IMU factor; when GNSS signal is normally received, the input of the factor graph construction subunit is pre-integrated IMU factor, GNSS measurement value and initial state prior information, when GNSS signal is not normally received, the input of the factor graph construction subunit is pre-integrated IMU factor, position increment and velocity increment predicted by the Transformer-LSTM hybrid neural network module and initial state prior information, and the output of the factor graph construction subunit is a complete factor graph model; the input of the adaptive weight adjustment subunit is GNSS measurement residual, and the output is adjusted GNSS factor weight; the input of the nonlinear optimization subunit is the factor graph model with weight, and the output is optimal navigation solution; The first cooperative work module is configured to acquire INS data when GNSS signals are normally received, and transmit the data to a data preprocessing subunit of a Transform-LSTM hybrid neural network module, acquire GNSS measurement values collected by a GNSS receiver, and transmit the GNSS measurement values to a factor graph construction subunit of the INS / GNSS integrated navigation module, output predicted position increments and velocity increments after processing the INS data by the Transform-LSTM hybrid neural network module, compare the predicted position increments and velocity increments with real GNSS position increments and velocity increments, train network parameters online, and thus update the Transform-LSTM hybrid neural network module; the INS / GNSS integrated navigation module is configured to pre-integrate IMU data, construct a factor graph in combination with GNSS measurement values, and obtain an optimal navigation solution through adaptive weight adjustment and nonlinear optimization; The second cooperative work module is configured to acquire INS data when GNSS signals are not normally received, and transmit the INS data to a Transform-LSTM hybrid neural network module to output predicted position increments and velocity increments, and the INS / GNSS integrated navigation module is configured to construct a factor graph and adjust weights according to IMU pre-integrated factors and the predicted position increments and velocity increments, and obtain an optimal navigation solution through nonlinear optimization.

9. An electronic device, comprising: Comprise: a processor, and a memory connected to the processor in communication; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to realize the GNSS / INS integrated navigation method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to realize the GNSS / INS integrated navigation method according to any one of claims 1-7.

Images