LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method

Abstract

The invention relates to an LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method, belongs to the technical field of multi-sensor integrated navigation and neural networks, and solves the problem of fault detection of an integrated navigation system. The method comprises the following steps: after carrying out structured preprocessing on an extracted Kalman filtering innovation sequence of the INS / GNSS / CNS integrated navigation system, inputting the Kalman filtering innovation sequence into a liquid neural network LNN to carry out time sequence modeling to obtain a hidden state sequence; performing GAFC and LVM two-dimensional image conversion on the hidden state sequence to obtain a GASF two-dimensional image, a GADF two-dimensional image and an LVM two-dimensional image; carrying out feature extraction on three-channel image data formed by GASF, GADF and LVM two-dimensional images by adopting a CNN network, and introducing a channel-level self-attention mechanism to carry out weighted fusion so as to obtain a fusion feature vector; and performing fault classification on the fusion feature vector by adopting a full-connection classifier, and performing supervised learning in combination with Softmax probability output and cross entropy loss so as to identify multiple types of faults in the integrated navigation system. According to the invention, the accuracy and robustness of fault detection of the integrated navigation system can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of multi-sensor integrated navigation and neural network, and particularly relates to a GNSS / INS / CNS integrated navigation system fault detection method based on LNN-CNN. BACKGROUND

[0002] In the prior art, with the wide development of high-altitude flight tasks in the fields of environmental monitoring, communication relay, safety warning, etc., high-altitude unmanned aerial vehicles (UAVs) are gradually becoming key high-altitude platforms in high-altitude persistent observation and communication support tasks due to their strong flexibility, reusability, and high platform stability. In order to ensure the persistent and stable operation of the UAVs in complex airspace environments, the precision and reliability of the navigation system are crucial.

[0003] Single navigation sensors have limitations in precision, coverage range, anti-interference, and environmental adaptability, and are difficult to meet the requirements of high-altitude UAVs for high reliability and high stability in complex working conditions. Generally, multi-sensor integrated navigation is needed to improve the precision, stability, and fault tolerance of the system in complex environments.

[0004] When such high-altitude UAVs perform all-weather and real-time information collection and transmission tasks, they need to operate stably for a long time in complex high-altitude environments. The integrated navigation system is susceptible to sensor drift, environmental interference, and equipment aging, and has multiple potential failure risks. Therefore, fault detection technology is an important support means to ensure the reliability of the integrated navigation system and flight safety. Through continuous monitoring and intelligent diagnosis of key parameters of the navigation system, timely warnings can be made before failures occur, and the integrated navigation precision and system redundancy can be effectively improved, thereby ensuring the persistent and stable operation of the high-altitude UAVs in complex task environments. For this reason, fault detection has become a key technology to ensure the reliability and safety of the integrated navigation system of high-altitude UAVs.

[0005] Data-driven intelligent methods have attracted widespread attention in the field of navigation fault detection. Compared with traditional detection methods that rely on physical modeling, data-driven methods have the advantages of flexible modeling and strong adaptability.

[0006] Therefore, by integrating the advantages of model-driven and data-driven methods, a "weak model-dependent" fault detection framework can be established to improve the detection accuracy and robustness in multiple complex fault scenarios. SUMMARY

[0007] In view of the above analysis, the application aims to disclose a GNSS / INS / CNS integrated navigation system fault detection method based on LNN-CNN; through a "weak model dependent" fault detection framework that fuses the advantages of model driven and data driven, the accuracy and robustness of fault detection are improved.

[0008] The GNSS / INS / CNS integrated navigation system fault detection method based on LNN-CNN disclosed by the application comprises the following steps:

[0009] Extracting a new information sequence comprising position and attitude information obtained by Kalman filtering of an INS / GNSS / CNS integrated navigation system;

[0010] Structuring and preprocessing the new information sequence to construct a training sample, inputting the training sample into a liquid neural network (LNN) to perform time series modeling and obtaining an implicit state sequence comprising new information response and new information historical dynamic evolution characteristics of the current moment;

[0011] Converting the implicit state sequence into GASF, GADF and LVM two-dimensional images respectively;

[0012] Extracting features of three-channel image data composed of the GASF, GADF and LVM two-dimensional images by using a CNN network, and introducing a channel-level self-attention mechanism to weight and fuse different image encoding channels to obtain a fusion feature vector;

[0013] Using a fully connected classifier to classify the fusion feature vector, combining Softmax probability output and cross-entropy loss for supervised learning to identify multiple faults in the INS / GNSS / CNS integrated navigation system.

[0014] Further, the observation equation of Kalman filtering of the INS / GNSS / CNS integrated navigation system is composed of absolute quantities provided by GNSS and CNS, wherein the expression of the observation model is:

[0015] Z=HX+V

[0016] In the formula, H is the observation matrix of the system, V is the measurement noise matrix, and the observation Z is composed of GNSS and CNS. GNSS Z CNS ] T Among the observation, GNSS provides the observation of position error Z GNSS =δp GNSS-INS =[δp x δp y δp z ] T , δp x , δp y , δpz respectively, are the three-axis position errors of the GNSS system relative to the INS system; the CNS provides observations of attitude errors Z CNS CNS-INS x y z T x y z respectively, are the three-axis attitude errors of the CNS relative to the INS system.

[0017] Further, in the structured preprocessing of the innovation sequence to construct the training sample, the innovation sequence is segmented in a sliding window manner; the training sample construction process includes:

[0018] 1) Set the sliding window length T, and each sample is the innovation sequence of the continuous T time points;

[0019] 2) Set the sliding step S to control the degree of overlap between samples;

[0020] 3) Starting from the starting index t0=0, the innovation time sequence is sequentially intercepted, and the mth sample is:

[0021]

[0022] Where, t m =(m-1)·S+1;

[0023] 4) For each innovation sample sequence X (m) , a label y (m) is generated accordingly; when the innovation sample sequence X (m) has any fault label, the entire sequence is marked as “fault”;

[0024] 5) Organize the sample set into a three-dimensional tensor input structure of the liquid neural network LNN And Where,

[0025] D is the dimension of the innovation at each time point, T is the length of the innovation sequence, and B is the number of samples per batch.

[0026] The jth innovation component at time t in the mth sample The label of the jth innovation component at time t in the mth sample

[0027] Further, in the liquid neural network LNN, the evolution of the hidden state h i (t) of the ith neuron is:

[0028] ​​​​​​​​

[0029] where h i (t) is the hidden state value of the i-th neuron at time t; τ i is the time constant of the i-th neuron; σ(·) is the Sigmoid function; is the j-th innovation component of the m-th sample at time t; is the weight of the input to hidden state connection; is the recurrent connection weight between hidden states; b i is the neuron bias;

[0030] Taking Δt = 1 as the unit sampling interval, the evolution of the hidden state value is:

[0031]

[0032]

[0033] where J is the total dimension of the input innovation; K is the number of hidden neurons; is the total weighted input representing the i-th neuron at time t, which is determined by the input vector, hidden state vector and bias term;

[0034] After LNN processing, the hidden state sequence of the m-th sample is:

[0035]

[0036] where H is the dimension of the hidden state.

[0037] Further, the GAFC two-dimensional image conversion process of the hidden state sequence includes:

[0038] 1) Normalizing the hidden state in the hidden state sequence;

[0039] to the l-th state channel is normalized to [-1, 1];

[0040]

[0041] where n is the n-th feature component of the l-th state channel; n ∈ [0, T-1];

[0042] 2) Introducing angle mapping to the normalized hidden state

[0043] 3) Converting the amplitude change of the normalized hidden state sequence into geometric structure in the angle space to obtain GASF and GADF two-dimensional images;

[0044] wherein,

[0045] In the GASF two-dimensional image, the point

[0046] In the GADF two-dimensional image, the point

[0047] wherein, r, s ∈ [0, T-1] are the r-th time and the s-th time in the channel.

[0048] Further, the LVM two-dimensional image conversion process on the hidden state sequence comprises:

[0049] 1) For each time point t1, set the window length as ω, and extract the local sequence in the hidden state sequence with t1 as the center:

[0050]

[0051] wherein, the missing boundary is filled by mirror processing;

[0052] 2) Calculate the local variance in the window:

[0053]

[0054] wherein, is the mean value of the local window. The calculated local variance is:

[0055] v = [v0, v1, …, v T-1 ]

[0056] 3) Use the square root product to construct the local disturbance intensity joint matrix:

[0057]

[0058] wherein, r, s ∈ [0, T-1] are the r-th time and the s-th time in the channel, v r , v s are the local variances at the r-th time and the s-th time.

[0059] Further, the CNN network comprises a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer and an output layer; wherein,

[0060] The first convolutional layer is used for extracting local features by using a plurality of 3x3 convolutional kernels on the input three-channel image tensor respectively, and extracting a first convolutional layer output feature map.

[0061] The first convolutional layer output feature map F1 = ReLU(W1·X img +b1);

[0062] wherein, Re LU(·) is a nonlinear activation function, is a three-channel image tensor; is a weight, and C1=16; is a bias term;

[0063] The first pooling layer is a max pooling layer, which is used to down-sample the output feature map after the first layer of convolution with a 2x2 window, and output a first pooling layer output feature map that retains a significant activation region;

[0064] The first pooling layer output feature map F2=MaxPool(F1, kernel=2, stride=2);

[0065] wherein,

[0066] The second convolutional layer is used to further extract and encode the output image features after down-sampling, and extract a second convolutional layer output feature map;

[0067] The second convolutional layer output feature map F3=ReLU(W2·F2+b2);

[0068] wherein, is a weight, and C2=32; is a bias term;

[0069] The second pooling layer is a max pooling layer, which is used to down-sample the output feature map of the second convolutional layer, compress the spatial dimension, and extract a stable second layer pooling output feature map;

[0070] The second layer pooling output feature map F4=MaxPool(F3, kernel=2, stride=2);

[0071] wherein,

[0072] The GASF, GADF and LVM two-dimensional images included in the second layer pooling output feature map F4 are respectively flattened and feature fused, and a fused feature vector is output.

[0073] Further, a channel-level self-attention mechanism is introduced in the output layer for feature fusion; the process includes:

[0074] 1) The second layer pooling output feature map F4 is flattened into three groups of vectors corresponding to the GASF, GADF and LVM two-dimensional images respectively to form a feature set wherein, d is the flattened feature dimension extracted by each channel CNN;

[0075] 2) Construct query Query, key Key and value Value for the set respectively:

[0076]

[0077] wherein, is a learnable weight matrix;

[0078] 3) Calculate attention score

[0079]

[0080] 4) Obtain attention-weighted channel feature output:

[0081]

[0082] 5) Average pooling on channel dimension is performed on the attention output to obtain a fusion feature vector:

[0083]

[0084] Further, the fault classification process in the full connection classifier for the fusion feature vector includes:

[0085] 1) The full connection layer maps the abstract feature to the specific class space, and the Softmax function is used to normalize the fusion feature vector into a multi-class probability output

[0086]

[0087] wherein, is a weight matrix, which learns the mapping relationship from the CNN feature space to the class space; is a bias term, and C is the number of classes; is a prediction probability vector for each class;

[0088] 2) A classification threshold mechanism is introduced to determine the judgment strategy by setting a decision threshold;

[0089]

[0090] wherein, the decision threshold τ∈(0,1).

[0091] Further, the cross entropy ι cel as the optimization objective of supervised learning is:

[0092]

[0093] wherein, B=32 is the batch size, C=2 is the number of fault classes, is the probability of the pth sample for the qth class prediction.

[0094] The present application can realize one of the following beneficial effects:

[0095] The disclosed LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method fully fuses the historical dynamic evolution characteristics of the innovation by extracting the innovation sequence obtained by Kalman filtering and using liquid neural network LNN for time series modeling, thereby more accurately capturing the time sequence variation law of the system state and providing a more comprehensive information basis for subsequent fault detection; the hidden state sequence is respectively converted into GAFC and LVM two-dimensional images, forming three-channel image data containing GASF, GADF and LVM two-dimensional images, and then combining the feature extraction capability of the convolutional neural network CNN and the channel-level self-attention mechanism for weighted fusion of different channel features, effectively integrating various feature information and significantly improving the accuracy of fault detection and reducing missed detection and false detection. In addition, the fully connected classifier is combined with the Softmax probability output and cross-entropy loss for supervised learning, which can accurately identify multiple faults in the system and meet the needs of complex system multi-fault type detection.

[0096] The method fully utilizes the information in the innovation sequence, improves the data utilization efficiency, and at the same time enhances the reliability and safety of the system, can timely discover faults and take measures to avoid system performance degradation or failure, and has good adaptability and universality, and can be widely applied to different types of INS / GNSS / CNS integrated navigation systems, providing a strong guarantee for the safe and stable operation of the navigation system. BRIEF DESCRIPTION OF DRAWINGS

[0097] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application.

[0098] Figure 1 The LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method in the embodiments of the present application is shown in the flowchart. DETAILED DESCRIPTION

[0099] The preferred embodiments of the present application will be specifically described below in conjunction with the drawings, wherein the drawings constitute a part of this application and are used together with the embodiments of the present application to explain the principles of the present application.

[0100] One embodiment of the present application discloses an LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method, as shown in Figure 1 The method comprises the following steps:

[0101] Step S1, extracting the innovation sequence including position and attitude information obtained by Kalman filtering of the INS / GNSS / CNS integrated navigation system;

[0102] Step S2, structuring the innovation sequence for pre-processing to construct a training sample, inputting to the liquid neural network LNN for time series modeling to obtain an implicit state sequence including innovation response and innovation historical dynamic evolution characteristics of the current time;

[0103] Step S3, respectively performing GAFC and LVM two-dimensional image conversion on the implicit state sequence to obtain GASF, GADF and LVM two-dimensional images;

[0104] Step S4, using the CNN network to extract features from the three-channel image data composed of GASF, GADF and LVM two-dimensional images, and introducing a channel-level self-attention mechanism to weight and fuse different image encoding channels to obtain a fusion feature vector;

[0105] Step S5, using a fully connected classifier to classify the fusion feature vector, combining the Softmax probability output and cross-entropy loss for supervised learning to identify multiple faults in the INS / GNSS / CNS integrated navigation system.

[0106] Specifically, in step S1, a centralized Kalman fusion filtering framework is used to construct system state equations and observation equations.

[0107] Through the error model, a 15-dimensional Kalman filter is designed, and the system state equation is:

[0108]

[0109] In the formula,

[0110] including three-axis misalignment angle error three-axis velocity error δv x , δv y , δv z , three-axis position error δL, δλ, δh, three-axis gyro drift ε x , ε y , ε z , three-axis accelerometer bias

[0111] In the formula, F is the state transition matrix of the system, G is the system noise driving matrix, and W is the system noise matrix.

[0112] The observation equation of the Kalman filter of the INS / GNSS / CNS integrated navigation system is composed of absolute quantities provided by GNSS and CNS, and the expression of the observation model is:

[0113] Z = HX + V

[0114] In the formula, H is the observation matrix of the system, V is the measurement noise matrix,

[0115] The observation Z = [Z GNSS Z GNS ] T , wherein the observation, GNSS provides the observation of the position error Z GNSS = δp GNSS -INS = [δp x δp y δp z ] T , δp x , δp y , δp z respectively, the three-axis position error of the GNSS system relative to the INS system; CNS provides the observation of the attitude error Z CNS = θ CNS-INS = [θ x θ y θ z ] T ; θ x , θ y , θ z respectively, the three-axis attitude error of the CNS relative to the INS system.

[0116] At time k, the innovation equation is:

[0117]

[0118] In the formula, is one-step prediction of the system state.

[0119] Reflects the response of the system to the observation error of each sensor at the current time, and is the key input for subsequent fault detection.

[0120] Specifically, in order to meet the needs of the LNN model for fixed-length time sequence input, in step S2, the innovation sequence is structured and preprocessed to construct the training sample, and the sliding window method is used to segment the innovation sequence; The process of constructing the training sample includes:

[0121] 1) Set the sliding window length T, and each sample is the innovation sequence of continuous T time points;

[0122] 2) Set the sliding step S to control the degree of overlap between samples;

[0123] 3) Start from the starting index t0=0, and sequentially intercept the innovation time sequence, the mth sample is:

[0124]

[0125] where t m = (m - 1) · S + 1;

[0126] 4) For each new information sample sequence X (m) , a label y (m) is generated; when there is any fault label in the new information sample sequence X (m) , the entire sequence is marked as "fault" (i.e. y = 1);

[0127] 5) To improve the model training speed, the sample set is organized into a three-dimensional tensor input structure of liquid neural network LNN and where,

[0128] D is the dimension of new information at each time point, T is the length of the new information sequence, and B is the number of samples per batch;

[0129] the jth new information component at the tth time point in the mth sample the label of the jth new information component at the tth time point in the mth sample

[0130] After the extraction and structured preprocessing of the new information sequence in the INS / GNSS / CNS integrated navigation system, the system obtains time series input data that can reflect the change pattern of the navigation state prediction error. This data is continuous in the time dimension, coupled between dimensions, and can reflect the instantaneous deviation characteristics of the navigation system under different observation conditions. To improve the modeling ability of the model for dynamic fault patterns, a liquid neural network (LNN) is used to model the time series of the new information sequence.

[0131] Liquid neural network (LNN) is a class of differential-driven time series modeling neurons that can simulate the change process of navigation new information with continuous time dynamics. LNN drives by approximating continuous time differential equations;

[0132] In liquid neural network LNN, the evolution of the hidden state h i (t) of the ith neuron is:

[0133]

[0134] where h i (t) is the hidden state value of the ith neuron at time t; τ i is the time constant of the ith neuron; σ(·) is usually Sigmoid; is the jth new information component at the tth time point in the mth sample; is the weight input to the hidden state connection; is the recurrent connection weight between hidden states; b i is the neuron bias;

[0135] The evolution of the hidden state value is

[0136]

[0137]

[0138] where J is the total dimension of the input innovation; K is the number of hidden neurons; is the total weighted input of the i-th neuron at time t, which is determined by the input vector, the hidden state vector and the bias term.

[0139] After LNN processing, the hidden state sequence of the m-th sample is

[0140]

[0141] where H is the dimension of the hidden state.

[0142] After completing the LNN-based hidden state time series modeling, the system obtains the hidden state sequence containing the dynamic evolution characteristics of the navigation innovation history. This hidden state sequence not only encodes the innovation response at the current time, but also contains the time series correlation and channel coupling relationship of the potential evolution process of the fault.

[0143] To further mine its spatial structure characteristics and improve the fault recognition accuracy by using the image discrimination ability of CNN, the hidden state sequence needs to be converted into a two-dimensional image form with spatial structure.

[0144] Specifically, in step S3, GAFC is used to convert the hidden state sequence output based on LNN into a two-dimensional image, so that the original time series data is encoded into a structured matrix through angle mapping and triangular transformation.

[0145] wherein the GAFC two-dimensional image conversion process of the hidden state sequence includes:

[0146] 1) Normalizing the hidden state in the hidden state sequence;

[0147] The l-th state channel is normalized to [-1, 1];

[0148]

[0149] where n is the n-th feature component of the l-th state channel; n ∈ [0, T-1];

[0150] 2) Introducing angle mapping to the normalized hidden state

[0151] By introducing angle mapping to the hidden state, the global correlation and local difference between different time points can be effectively expressed, and the ability of CNN to capture fault evolution space patterns can be enhanced,

[0152] 3) The amplitude change of the normalized hidden state sequence is converted into a geometric structure in the angle space to obtain GASF and GADF two-dimensional images; the originally one-dimensional dynamic change is reflected in the form of angle and difference in the two-dimensional matrix;

[0153] wherein,

[0154] In the GASF two-dimensional image, the point

[0155] In the GADF two-dimensional image, the point

[0156] wherein, r, s ∈ [0, T-1] is the rth time and the sth time in the channel.

[0157] In addition to the "global trend" and "angle difference" represented by GASF and GADF, in order to further enhance the perception ability of the model to local mutation characteristics, LVM is introduced as the third channel of image input, which extracts the variation degree of state vector in each time period, so as to achieve high sensitivity to sudden faults (such as transient drift) and slowly changing faults (such as sensor aging).

[0158] Specifically, the LVM two-dimensional image conversion process of the hidden state sequence includes:

[0159] 1) For each time point t1, set the window length as ω centered on it, and extract the local sequence in the hidden state sequence:

[0160]

[0161] Wherein, the missing boundary is filled by mirror processing;

[0162] 2) Calculate the local variance in the window:

[0163]

[0164] In the formula, is the mean value of the local window. The calculated local variance is:

[0165] v = [v0, v1, …, v T-1 ]

[0166] 3) Use the square root product to construct the local disturbance intensity joint matrix:

[0167]

[0168] where r, s ∈ [0, T-1] are the r-th and s-th time in the channel, v r , v s are the local variance of the r-th and s-th time.

[0169] In order to capture the complete system dynamic characteristics from the innovation and better adapt the two-dimensional image conversion module and the CNN network layer structure, the innovation sequence T = 50 is taken.

[0170] By mapping the hidden state sequence output by the LNN into the GASF, GADF and LVM two-dimensional image structure, the system successfully completes the spatial expression conversion of the time series data. The image not only retains the trend information and mutation characteristics in the original time series, but also strengthens the expression ability of the cooperative abnormal mode between channels through the multi-channel structure. Next, the CNN is introduced for feature extraction, and multi-level convolution operation and spatial feature learning are performed on the three-channel image to extract the local high response area and global discriminant structure related to the fault, further enhancing the model's ability to distinguish different types of fault patterns.

[0171] Specifically, in step S4, the CNN network used includes a first convolutional layer, a first pooling layer, a second convolutional layer, a second pooling layer, and an output layer.

[0172] The first convolutional layer is used to extract local features of the input three-channel image tensor (GASF, GADF, LVM) using multiple 3x3 convolution kernels, and to extract a first convolutional layer output feature map.

[0173] The first convolutional layer output feature map F1 = ReLU(W1·X img +b1);

[0174] where Re LU(·) is a nonlinear activation function, is a three-channel image tensor; is a weight, and C1 = 16; is a bias term;

[0175] In the two-dimensional convolution operation of the first convolutional layer, the ReLU nonlinear activation function is combined to effectively improve the network's response ability to edge mutations, local disturbance textures in the fault area.

[0176] The first pooling layer is a max pooling layer, which is used to downsample the output feature map after the first convolutional layer with a 2x2 window, outputting a first pooling layer output feature map that retains significant activation regions while reducing spatial dimensions and computational complexity.

[0177] The first pooling layer outputs a feature map F2 = MaxPool(F1, kernel = 2, stride = 2)

[0178] wherein,

[0179] The down-sampling process of the first pooling layer enhances the fault tolerance of the model under small-scale translation and scale transformation, and helps to improve the robustness of the fault detection model.

[0180] The second convolutional layer is used to further extract and encode the down-sampled image features, and extracts a second convolutional layer output feature map F3 = Re LU(W2·F2+b2).

[0181] The second convolutional layer output feature map F3 = Re LU(W2·F2+b2).

[0182] wherein, is a weight, and C2 = 32; is a bias term;

[0183] The second convolutional layer is used to further extract and encode the down-sampled image features, and through the ReLU activation to continue to introduce non-linear expression, to ensure that the network has good discrimination ability for fault evolution process.

[0184] The second pooling layer is a max-pooling layer, which is used to down-sample the second convolutional layer output feature map, compress the spatial dimension, and extract a stable second layer pooling output feature map.

[0185] The second layer pooling output feature map F4 = MaxPool(F3, kernel = 2, stride = 2).

[0186] wherein,

[0187] The GASF, GADF and LVM two-dimensional images included in the second layer pooling output feature map F4 are respectively flattened and feature fused, and a fused feature vector is output.

[0188] In the second pooling layer, the recognition ability of the model for large-scale structural disturbances (such as systematic fault evolution path) in the image is further enhanced through feature aggregation.

[0189] Since the input image is composed of GASF, GADF and LVM in three different encoding modes, each channel has heterogeneity and bias when responding to sudden or gradual fault patterns. If they are directly spliced for classification, the relative importance of different channels may be ignored. To further enhance the adaptive ability of the model in feature fusion between channels, a channel-level self-attention mechanism is introduced at the output stage of the CNN, which automatically captures the most discriminative feature channels through learning weight allocation, and enhances the reliability of the final fault classification decision.

[0190] Specifically, the feature fusion is performed in the output layer by introducing a channel-level self-attention mechanism; the process includes:

[0191] 1) The spatial-level feature F4 is flattened into three groups of vectors corresponding to the GASF, GADF and LVM two-dimensional images respectively to form a feature set where d is the dimension of the flattened feature extracted by each channel CNN;

[0192] 2) Query, Key and Value are constructed for the set respectively:

[0193]

[0194] where is a learnable weight matrix;

[0195] 3) The attention score is calculated

[0196]

[0197] 4) The channel feature output after attention weighting is obtained:

[0198]

[0199] 5) The attention output is averaged-pooled in the channel dimension to obtain a fusion feature vector:

[0200]

[0201] After completing the multi-level spatial feature extraction of the image-form input, the system has obtained a highly compressed and discriminative fault representation vector. To realize intelligent identification of the running state of the navigation system, in step S5, a fully connected classifier and a fault discrimination mechanism are introduced to perform multi-class fault classification on the attention-fused vector.

[0202] The fault classification process in the fully connected classifier for classifying the fusion feature vector includes:

[0203] 1) The fully connected layer maps the abstract feature to a specific class space, and the Softmax function is used to normalize the fusion feature vector into a multi-class probability output

[0204]

[0205] where is a weight matrix, which learns the mapping relationship from the CNN feature space to the class space; is a bias term, and C is the number of classes; The prediction probability vector of each category is obtained.

[0206] 2) A classification threshold mechanism is introduced to determine the judgment strategy by setting a decision threshold.

[0207]

[0208] wherein the decision threshold τ ∈ (0, 1).

[0209] 1) The full connection layer maps the abstract features to the specific category space, and the Softmax function is used to normalize the fusion feature vector into a multi-class probability output

[0210]

[0211] wherein, is a weight matrix, which learns the mapping relationship from the CNN feature space to the category space; is a bias term, and C is the number of categories; The prediction probability vector of each category is obtained.

[0212] The probability distribution of the Softmax output can represent the relative probability of fault classification, but sometimes the confidence distribution is close, which is easy to misjudge. Therefore, a classification threshold mechanism is introduced to enhance the judgment reliability: the prediction probability vector of each category is expressed as:

[0213]

[0214] 2) A classification threshold mechanism is introduced to determine the judgment strategy by setting a decision threshold.

[0215]

[0216] wherein the decision threshold τ ∈ (0, 1).

[0217] The network uses cross-entropy as the optimization target of supervised learning, and the loss function shows excellent convergence speed and stability in the multi-class fault recognition scene, which well measures the difference between the predicted probability distribution and the true label.

[0218] The cross-entropy ι as the optimization target of supervised learning is: cel

[0219]

[0220] wherein B = 32 is the batch size, C = 2 is the number of fault categories, is the probability of the pth sample to the qth category.

[0221] ​Combined with the supervision label in the model training stage, the stage realizes automatic determination of sudden faults and slowly changing faults, and completes an end-to-end system closed loop from navigation innovation perception, dynamic modeling, image conversion, spatial feature extraction to finally introducing a self-attention mechanism intelligent fault recognition.

[0222] In summary, the method of the embodiment of the application is aimed at the technical bottleneck that the integrated navigation system (INS / GNSS / CNS) is prone to sudden and slowly changing fault interference in a complex environment, and the traditional method is difficult to simultaneously consider real-time and multi-fault identification ability;

[0223] In the method of the embodiment, the innovation sequence obtained by Kalman filtering is extracted and time series modeling is performed using a liquid neural network (LNN), which fully integrates the historical dynamic evolution characteristics of the innovation, thereby more accurately capturing the time series variation law of the system state and providing a more comprehensive information basis for subsequent fault detection; the hidden state sequence is respectively converted into GAFC and LVM two-dimensional images, forming three-channel image data containing GASF, GADF and LVM two-dimensional images, and then combined with the feature extraction capability of a convolutional neural network (CNN) and the weighted fusion of different channel features by a channel-level self-attention mechanism, effectively integrating various feature information and significantly improving the accuracy of fault detection and reducing missed and false detections; in addition, a fully connected classifier is used for supervised learning in combination with Softmax probability output and cross-entropy loss, which can accurately identify multiple types of faults in the system and meet the needs of complex system multi-fault type detection.

[0224] The method of the embodiment fully utilizes the information in the innovation sequence, improves the data utilization efficiency, and at the same time enhances the reliability and safety of the system, can timely discover faults and take measures to avoid system performance degradation or failure, and has good adaptability and universality, and can be widely applied to different types of INS / GNSS / CNS integrated navigation systems, providing a strong guarantee for the safe and stable operation of the navigation system.

[0225] The above is only a preferred specific embodiment of the application, but the protection scope of the application is not limited thereto, and any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the application, which should be covered within the protection scope of the application.

Claims

1. A GNSS / INS / CNS integrated navigation system fault detection method based on LNN-CNN, characterized in that: The following steps are involved: Extract the new information sequence including position and attitude information obtained by Kalman filtering of INS / GNSS / CNS integrated navigation system; The innovation sequence is structured preprocessed to construct training samples, which are then input into the liquid neural network (LNN) for time series modeling to obtain a hidden state sequence that includes the innovation response at the current moment and the historical dynamic evolution characteristics of the innovation; Perform GAFC and LVM two-dimensional image conversion on the hidden state sequence to obtain GASF, GADF and LVM two-dimensional images; The CNN network is used to extract features from three-channel image data consisting of GASF, GADF and LVM two-dimensional images, and a channel-level self-attention mechanism is introduced to perform weighted fusion of different image coding channels to obtain a fused feature vector. A fully connected classifier is used to classify the fused feature vector for fault classification, and the Softmax probability output is combined with the cross entropy loss for supervised learning to identify multiple types of faults in the INS / GNSS / CNS integrated navigation system.

2. The GNSS / INS / CNS integrated navigation system fault detection method based on LNN-CNN according to claim 1, characterized in that: The observation equation of the Kalman filter of the INS / GNSS / CNS integrated navigation system is composed of the absolute quantities provided by GNSS and CNS. The expression of the observation model is: Z=HX+V Where H is the measurement matrix of the system, V is the measurement noise matrix, and the observation quantity Z = [Z GNSS Z CNS ] T , where GNSS provides the observation Z of position error in the observation GNSS =δp GNSS-INS =[δp x δp y δp z ] T ,δp x ,δp y ,δp z are the three-axis position errors of the GNSS system relative to the INS system; CNS provides the observation Z of the attitude error CNS =θ CNS-INS =[θ x θ y θ z ] T θ x ,θ y ,θ z are the three-axis attitude errors of CNS relative to INS system.

3. The LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method according to claim 2, characterized in that: In the process of constructing training samples by structured preprocessing of the new information sequence, the new information sequence is segmented by using a sliding window method; The process of constructing training samples includes: 1) Set the sliding window length to T, and each sample is a sequence of new information at T consecutive time points; 2) Set the sliding step size S to control the degree of overlap between samples; 3) Starting from the starting index t0=0, the new information time series is intercepted in sequence. The mth sample is: Among them, t m =(m-1)·S+1; 4) For each new information sample sequence X (m) , corresponding to the generation of a label y (m) ; When the new information sample sequence X (m) If any fault label exists, the entire sequence is marked as "faulty"; 5) Organize the sample set into a three-dimensional tensor input structure of the liquid neural network LNN and in, D is the dimension of innovation at each time point, T is the length of the innovation sequence, and B is the number of samples in each batch; The jth new information component at time t in the mth sample The label of the jth innovation component at time t in the mth sample 4. The LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method according to claim 3, characterized in that: In the liquid neural network LNN, the hidden state h of the i-th neuron is i The evolution of (t) is: Where h i (t) is the hidden state value of the i-th neuron at time t; τ i is the time constant of the i-th neuron; σ(·) is the Sigmoid function; is the jth new information component of the mth sample at the tth moment; is the weight of the connection from input to hidden state; is the recursive connection weight between hidden states; b i for neuronal bias; The unit sampling interval is Δt = 1. After the continuous differential equation is discretized using the Euler approximation method, the evolution of the hidden state value is: Where J is the total dimension of the input information; K is the number of hidden neurons; represents the total weighted input of the i-th neuron at time t, which is determined by the input vector, hidden state vector and bias term; After LNN processing, the hidden state sequence of the mth sample is: Where H is the hidden state dimension.

5. The LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method according to claim 4, characterized in that: The GAFC two-dimensional image conversion process for the hidden state sequence includes: 1) Normalize the hidden state in the hidden state sequence; For the lth state channel Normalized to [-1,1]; Where n is the nth characteristic component of the lth state channel; n∈[0,T-1]; 2) Introducing angle mapping to the normalized hidden state 3) Convert the amplitude change of the normalized hidden state sequence into the geometric structure in the angular space to obtain the GASF and GADF two-dimensional images; in, In the GASF two-dimensional image, point In GADF two-dimensional images, points Among them, r,s∈[0,T-1] are the rth time and the sth time in the channel.

6. The LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method according to claim 4, characterized in that: The LVM two-dimensional image conversion process for the hidden state sequence includes: 1) For each time point t1, take it as the center, set the window length to ω, and extract the local sequence in the hidden state sequence: Among them, the missing boundaries are processed by mirror filling; 2) Calculate the local variance within the window: Where, is the mean of the local window; the local variance is calculated as: v=[v0,v1,…,v T-1 ] 3) Use square root product to construct the local perturbation intensity joint matrix: Among them, r,s∈[0,T-1] is the rth time and the sth time in the channel, v r 、v s is the local variance at the rth moment and the sth moment.

7. The LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method according to any one of claims 1 to 6, characterized in that: The CNN network includes the first convolutional layer, the first pooling layer, the second convolutional layer, the second pooling layer and the output layer; among them, The first convolutional layer is used to extract local features of the input three-channel image tensor using multiple 3×3 convolution kernels to extract the output feature map of the first convolutional layer; The first convolutional layer outputs a feature map F1 = Re LU (W1·X img +b1); Among them, Re LU(·) is a nonlinear activation function, is a three-channel image tensor; is the weight, C1=16; is the bias term; The first pooling layer is a maximum pooling layer, which is used to downsample the output feature map after the first convolution layer with a 2×2 window, and output the output feature map of the first pooling layer that retains the significant activation area; The first pooling layer outputs the feature map F2 = MaxPool (F1, kernel = 2, stride = 2); in, The second convolutional layer is used to further extract and encode the features of the downsampled output image and extract the output feature map of the second convolutional layer; The second convolutional layer outputs the feature map F3 = Re LU (W2·F2+b2); in, is the weight, C2=32; is the bias term; The second pooling layer is a maximum pooling layer, which is used to downsample the output feature map of the second convolutional layer, compress the spatial dimension, and extract a stable second-layer pooling output feature map; The second layer pooling output feature map F4 = MaxPool (F3, kernel = 2, stride = 2); in, The GASF, GADF and LVM two-dimensional images included in the second-layer pooling output feature map F4 are flattened and feature fused respectively, and the fused feature vector is output.

8. The LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method according to claim 7, characterized in that: In the output layer, feature fusion is performed by introducing a channel-level self-attention mechanism; the process includes: 1) Flatten the second-layer pooling output feature map F4 into three sets of vectors corresponding to the GASF, GADF and LVM two-dimensional images respectively Constructing a feature set Where d is the flattened feature dimension extracted by each channel CNN; 2) Construct the query, key, and value for the collection: in, is a learnable weight matrix; 3) Calculate attention score 4) Obtain the channel feature output after attention weighting: 5) Perform average pooling on the attention output in the channel dimension to obtain the fused feature vector:

9. The LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method according to claim 8, characterized in that: The fault classification process in the fully connected classifier for fault classification of the fused feature vector includes: 1) The fully connected layer maps the abstract features to the specific category space and uses the Softmax function to normalize the fused feature vector into multi-class probability output in, is the weight matrix, learning the mapping relationship from CNN feature space to category space; is the bias term, C is the number of categories; Predict probability vectors for each category; 2) Introducing a classification threshold mechanism to determine the judgment strategy by setting the judgment threshold; Among them, the decision threshold τ∈(0,1).

10. The LNN-CNN-based GNSS / INS / CNS integrated navigation system fault detection method according to claim 9, characterized in that: Cross-entropy ι as an optimization objective for supervised learning cel for: Among them, B=32 is the batch size, C=2 is the number of fault categories, is the probability of the p-th sample predicting the q-th category.

Images