Navigation equipment real-time fault diagnosis method based on cross-modal attention fusion

Abstract

A navigation equipment real-time fault diagnosis method based on cross-modal attention fusion belongs to the technical field of power grid equipment fault diagnos.The method comprises the steps that firstly, the type of real-time diagnosis data capable of being obtained by navigation equipment is systematically analyzed, and the real-time diagnosis data is divided into four modals of sensor data, electrical data, acoustic data and text data; after data cleaning is completed, preliminary feature extraction is implemented. Then, constructing a fault diagnosis model based on a cross-modal attention mechanism, constructing a training data set by using the joint features, and carrying out neural network model training; after model training is completed, real-time fault diagnosis of the navigation equipment can be realized through a data pre-processing technology and a fault diagnosis model based on acquired data. According to the scheme, a multi-modal real-time sensing system is constructed, deep feature interaction is realized in combination with a cross-modal attention mechanism, and closed-loop diagnosis capability is established. And core guarantee is provided for real-time state sensing and accurate fault positioning of navigation equipment.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of power grid equipment fault diagnosis, and particularly relates to a navigation equipment real-time fault diagnosis method based on cross-modal attention fusion. BACKGROUND

[0002] As a key core component of transportation equipment, the operation reliability of the navigation equipment is directly related to user safety and business continuity. Traditional fault diagnosis relies on single-modal data analysis, which is difficult to deal with hidden faults under complex working conditions. Artificial inspection has a lag and cannot meet the real-time operation and maintenance requirements. Especially in high dynamic scenes, the rapid propagation characteristics of faults require a response speed of seconds. The existing methods have significant deficiencies in cross-modal feature fusion efficiency and diagnosis timeliness. SUMMARY

[0003] In order to solve the above problems, the application provides a navigation equipment real-time fault diagnosis method based on cross-modal attention fusion, which comprises the following steps:

[0004] S1, acquiring real-time diagnosis data;

[0005] S2, data cleaning;

[0006] S3, preliminary feature extraction;

[0007] S4, constructing a fault diagnosis model based on a cross-modal attention mechanism, using joint features to construct a training data set, and training a neural network model;

[0008] S5, after the model training is completed, the real-time fault diagnosis of the navigation equipment can be realized based on the collected data through data preprocessing technology and the fault diagnosis model.

[0009] In the step S1, the data types are divided into four modes of sensor data, electrical data, acoustic data and text data. The sensor data includes GNSS position data, GNSS speed data, IMU acceleration data, IMU angular velocity data, temperature data and humidity data. The electrical data includes current, voltage and power data. The acoustic data includes equipment operation noise and ultrasonic data. The text data is equipment operation log. The data collected at a fixed time interval constitutes a single sample. The original sample set DataSet1 is constructed by collecting samples in normal state and various fault states.

[0010] Further, in the step S2, the three types of numerical modal data of sensor data, electrical data and acoustic data in each sample are screened for abnormal values using the Z-score method, missing values and abnormal values are filled using linear interpolation, data smoothing filtering is performed using the Lowess method, and finally data standardization processing is completed through Min-Max normalization; the text type modal data is cleaned through the processing methods of removing special characters, removing punctuation marks, and removing pause words; all the cleaned data together form a cleaned sample set DataSet2.

[0011] Further, in the step S3, in the sensor data, the GNSS position data includes a longitude coordinate sequence S1 and a latitude coordinate sequence S2, the GNSS speed data includes an x-axis speed sequence S3, a y-axis speed sequence S4 and a z-axis speed sequence S5, the IMU acceleration data includes an x-axis acceleration sequence S6, a y-axis acceleration sequence S7 and a z-axis acceleration sequence S8, the IMU angular velocity data includes an x-axis angular velocity time sequence S9, a y-axis angular velocity time sequence S10 and a z-axis angular velocity time sequence S11, the temperature data is a temperature value sequence S12, and the humidity data is a humidity percentage sequence S13, a total of 13 dimensions of original data; the mean and standard deviation of S1 to S13 are extracted as two statistical features, the peak-to-peak value, kurtosis value, skewness value, waveform factor and pulse factor of S1 to S11 are extracted as five time domain features, and the average frequency, center frequency and frequency standard deviation of S1 to S11 after fast Fourier transform are extracted as three frequency domain features; a total of 114 dimensional features are extracted from the sensor data in each sample; dimensionality reduction is performed through PCA principal component analysis, and 15 dimensional reconstructed features with the highest contribution rate are retained;

[0012] In the electrical data, the current data is a direct current sequence E1, the voltage data is a direct voltage sequence E2, and the power data includes an active power sequence E3, a total of 3 dimensions of original data; the average value, standard deviation, peak-valley difference and change rate of S1, S2 and S3 are extracted as four time domain features, and the harmonic distortion rate, signal-to-noise ratio and peak-to-average ratio of S1, S2 and S3 after fast Fourier transform are extracted as three frequency domain features; a total of 21 dimensional features are extracted from the electrical data in each sample; dimensionality reduction is performed through PCA principal component analysis, and 8 dimensional reconstructed features with the highest contribution rate are retained;

[0013] In the acoustic data, the running noise data is a wideband noise sequence A1, and the ultrasonic wave data is a high-frequency noise sequence A2, which are 2-dimensional original data; 3 time domain features of energy value, kurtosis value and pulse factor are extracted from A1, 3 frequency domain features of mean frequency, center frequency and frequency standard deviation are extracted after fast Fourier transform of A1, 3 indexes of pulse count rate, pulse peak value and pulse average value are extracted after envelope demodulation analysis of A2, and 9-dimensional features are extracted from the acoustic data in each sample; the dimension is reduced by PCA principal component analysis, and the 4-dimensional reconstructed features with the highest contribution rate are retained;

[0014] In the text data, 4 text statistical features of byte number mean value, byte number variance, repeated log maximum length and log interval mean value are calculated for T1;

[0015] After the above process, the DataSet2 sample is converted into a feature sample set DataSet3 containing the reconstructed features of each modality.

[0016] Further, in the step S4, a fault diagnosis model is built, and the entire fault diagnosis model is built with an attention fusion mechanism as the core, and is composed of a feature isomorphism encoding layer, a modality calculation layer, a modality association layer and a modality fusion layer to form a complete architecture.

[0017] Further, in the feature isomorphism encoding layer, a linear fully connected encoder maps the original heterogeneous modality features to a unified dimensional space through four independent fully connected operations, specifically converts 15-dimensional sensor features, 8-dimensional electrical features, 4-dimensional acoustic features and 4-dimensional text features into 64-dimensional isomorphic representations, and establishes a standardized feature basis for subsequent processing.

[0018] Further, after entering the modality calculation layer, the system is processed in parallel in a target modality calculation path and a source modality calculation path; the target path generates a query matrix Q using a fully connected layer with an output of 128 dimensions, and the source path generates intermediate features through a fully connected layer with a dimension of 256, and equally divides the intermediate features into a key matrix K and a value matrix V; after each sample is processed by this layer, four complete Q, K and V matrix combinations are formed, corresponding to the four types of modalities.

[0019] Further, the modality association layer is a core interactive module of the model, and its multi-head attention mechanism drives the dynamic association calculation between modalities. This layer adopts a cyclic processing strategy, so that each modality takes turns as a target modality, and all modalities are input as source modalities, thereby forming sixteen combinations of four types of target modalities and four types of source modalities.

[0020] Each combination generates an association sequence Attn with a length of 128 through an attention mechanism, which completely captures the cross-modality interaction features:

[0021] The calculation formula of Attn is as follows:

[0022]

[0023] wherein Q is a target modality query matrix, K and V are source modality key matrix and value matrix, d k is a scaling factor, in the present patent d k is uniformly 128.

[0024] Further, the modality fusion layer adopts a gated neural network architecture to realize feature compression; sixteen groups of association sequences are first integrated into a 16x128 association matrix as input; a gated weight generation network learns the dynamic weight distribution between features through a fully connected layer, and outputs a 16x16 weight matrix; the weight matrix and the original association matrix are multiplied to realize weighted fusion, and the fusion result is compressed and expanded after adaptive pooling layer, forming a 64-dimensional one-dimensional sequence representing the essential features of the sample; this feature vector is directly connected to the fault classifier, and the mapping relationship with the fault type is established through end-to-end training, and finally the fault diagnosis task is completed.

[0025] The beneficial effects of the present application are: the present scheme constructs a multi-modal real-time perception system, realizes feature deep interaction by combining cross-modal attention mechanism, establishes closed-loop diagnosis capability, and provides core guarantee for real-time state perception and accurate fault positioning of navigation equipment. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a flow chart of the present application based on cross-modal attention fusion for real-time fault diagnosis method of navigation equipment;

[0027] Figure 2 is a data cleaning flow chart of the present application;

[0028] Figure 3 is a feature extraction flow chart of the present application. DETAILED DESCRIPTION

[0029] In order to make the technical means adopted by the present application and the purpose easy to understand, the present application is further described below in combination with specific embodiments, a real-time fault diagnosis method of navigation equipment based on cross-modal attention fusion, comprising the following steps:

[0030] S1, obtaining real-time diagnosis data;

[0031] S2, data cleaning;

[0032] S3, implementing preliminary feature extraction;

[0033] S4, constructing a fault diagnosis model based on cross-modal attention mechanism, using joint features to construct a training data set, and performing neural network model training;

[0034] S5, after completing the model training, the real-time fault diagnosis of the navigation device can be realized based on the collected data through data preprocessing technology and fault diagnosis model.

[0035] The patent is to realize the real-time fault diagnosis of the navigation device by using multi-modal data. The patent proposes a fault diagnosis method based on cross-modal attention fusion, as shown in Figure 1 The method first analyzes the types of real-time diagnosis data that can be obtained by the navigation device, divides them into four modes of sensor data, electrical data, acoustic data and text data, and performs preliminary feature extraction after data cleaning. Then, a fault diagnosis model based on cross-modal attention mechanism is constructed, a joint feature is used to construct a training data set, and a neural network model is trained. After completing the model training, the real-time fault diagnosis of the navigation device can be realized based on the collected data through data preprocessing technology and fault diagnosis model.

[0036] Technical route:

[0037] In step S1, the real-time data involved in the fault diagnosis of the navigation device includes: sensor data containing GNSS position data, GNSS speed data, IMU acceleration data, IMU angular velocity data, temperature data and humidity data; electrical data including current, voltage and power data; acoustic data covering device running noise and ultrasonic data; and text data being device running log. The data collected at fixed time intervals (such as 5 seconds) constitute a single sample, and the original sample set DataSet1 is constructed by collecting samples under normal state and various fault states.

[0038] In step S2, in view of the uneven quality of multi-source data, strict cleaning of the data is needed before feature extraction. The sensor, electrical and acoustic three types of numerical mode data in each sample are screened for abnormal values using Z-score method, missing values and abnormal values are filled by linear interpolation, data smoothing filter is implemented by Lowess method, and finally data standardization processing is completed by Min-Max normalization. The text type modal data is cleaned by special character removal, punctuation removal and pause word removal processing method. All the cleaned data together constitute the cleaned sample set DataSet2, and then differential feature extraction strategies are implemented according to the characteristics of each modal data.

[0039] In step S3, as shown in Figure 2As shown, in the sensor data, the GNSS position data includes a longitude coordinate sequence S1 and a latitude coordinate sequence S2, the GNSS speed data includes an x-axis speed sequence S3, a y-axis speed sequence S4, and a z-axis speed sequence S5, the IMU acceleration data includes an x-axis acceleration sequence S6, a y-axis acceleration sequence S7, and a z-axis acceleration sequence S8, the IMU angular velocity data includes an x-axis angular velocity time sequence S9, a y-axis angular velocity time sequence S10, and a z-axis angular velocity time sequence S11, the temperature data is a temperature value sequence S12, and the humidity data is a humidity percentage sequence S13, which are 13 dimensions of original data. The mean and standard deviation of S1 to S13 are extracted as two statistical features, the peak-to-peak value, kurtosis value, skewness value, waveform factor, and pulse factor of S1 to S11 are extracted as five time-domain features, and the average frequency, center frequency, and frequency standard deviation of S1 to S11 are extracted as three frequency-domain features after fast Fourier transform. A total of 114-dimensional features are extracted from the sensor data in each sample. Through PCA principal component analysis dimension reduction, 15-dimensional reconstructed features with the highest contribution rate are retained.

[0040] In the electrical data, the current data is a direct current sequence E1, the voltage data is a direct voltage sequence E2, and the power data includes an active power sequence E3, which are 3 dimensions of original data. The average value, standard deviation, peak-valley difference, and change rate of S1, S2, and S3 are extracted as four time-domain features, and the harmonic distortion rate, signal-to-noise ratio, and peak-to-average ratio of S1, S2, and S3 are extracted as three frequency-domain features after fast Fourier transform. A total of 21-dimensional features are extracted from the electrical data in each sample. Through PCA principal component analysis dimension reduction, 8-dimensional reconstructed features with the highest contribution rate are retained.

[0041] In the acoustic data, the operating noise data is a wideband noise sequence A1, and the ultrasonic data is a high-frequency noise sequence A2, which are 2 dimensions of original data. The energy value, kurtosis value, and pulse factor of A1 are extracted as three time-domain features, the average frequency, center frequency, and frequency standard deviation of A1 are extracted as three frequency-domain features after fast Fourier transform, the pulse count rate, pulse peak-to-peak value, and pulse average value of A2 are extracted as three indicators after envelope demodulation analysis, and a total of 9-dimensional features are extracted from the acoustic data in each sample. Through PCA principal component analysis dimension reduction, 4-dimensional reconstructed features with the highest contribution rate are retained.

[0042] In the text data, only the device operation log T1 is considered, and the byte number mean, byte number variance, repeated log maximum length, and log interval mean are calculated as four text statistical features of T1, without the need for other processing.

[0043] After the above process, the DataSet2 sample is converted into a feature sample set DataSet3 containing the reconstruction features of each modality, which fully retains the key information of the equipment operating state and provides high-quality training input for the subsequent fault diagnosis model.

[0044] In the step S4, as shown in Figure 3 Next, the fault diagnosis model is built. The entire fault diagnosis model is built with an attention fusion mechanism as the core and consists of a feature isomorphism encoding layer, a modality calculation layer, a modality association layer, and a modality fusion layer to form a complete architecture.

[0045] In the feature isomorphism encoding layer, a linear fully connected encoder maps the original heterogeneous modality features to a unified dimensional space through four independent fully connected operations. Specifically, the 15-dimensional sensor features, 8-dimensional electrical features, 4-dimensional acoustic features, and 4-dimensional text features are respectively converted into 64-dimensional isomorphic representations to establish a standardized feature basis for subsequent processing.

[0046] After entering the modality calculation layer, the system is divided into a target modality calculation path and a source modality calculation path for parallel processing. The target path uses a fully connected layer with an output of 128 dimensions to generate a query matrix Q, while the source path generates intermediate features through a fully connected layer with a dimension of 256 and equally divides them into a key matrix K and a value matrix V. Each sample forms a complete combination of Q, K, and V matrices after processing through this layer, corresponding to the four types of modalities.

[0047] The modality association layer is the core interaction module of the model, and its multi-head attention mechanism drives the dynamic association calculation between modalities. This layer uses a cyclic processing strategy, which makes each modality take turns as the target modality while considering all modalities (including the current target modality itself) as source modality input. Thus, it forms sixteen combinations of four types of target modalities and four types of source modalities, as shown in the following table.

[0048] Table 1: Source Modality-Target Modality Combination Table

[0049]

[0050]

[0051] Each combination generates an association sequence Attn with a length of 128 through the attention mechanism, which fully captures the cross-modality interaction features.

[0052] The calculation formula of Attn is as follows:

[0053]

[0054] where Q is the target modality query matrix, K and V are the source modality key matrix and value matrix, d k is a scaling factor, and dk is unified to 128.

[0055] The modal fusion layer adopts a gated neural network architecture to realize feature compression. The sixteen sets of correlation sequences are first integrated into a 16x128 correlation matrix as input. The gated weight generation network learns the dynamic weight distribution between features through a fully connected layer and outputs a 16x16 weight matrix. The weight matrix is multiplied with the original correlation matrix to realize weighted fusion. The fusion result is compressed and expanded through an adaptive pooling layer to form a 64-dimensional one-dimensional sequence representing the essential features of the sample. This feature vector is directly connected to the fault classifier to establish a mapping relationship with the fault type through end-to-end training, and finally complete the fault diagnosis task.

[0056] The entire model realizes intelligent fusion of multi-source heterogeneous data and training of a diagnosis model through hierarchical feature conversion and modal interaction mechanism.

[0057] Finally, the real-time acquired multi-modal data of the navigation device can be transmitted into the trained intelligent diagnosis model after data preprocessing and feature extraction process. The model performs inference analysis based on the input features, outputs fault type recognition, and realizes real-time fault diagnosis function of the navigation device as a whole.

[0058] The above describes only the preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can make equivalent replacements or changes to the technical solutions and concepts of the present application within the scope of the disclosed technology, which should be covered within the protection scope of the present application.

Claims

1. A navigation equipment real-time fault diagnosis method based on cross-modal attention fusion, characterized in that, The method comprises the following steps: S1, obtaining real-time diagnosis data; S2, data cleaning; S3, implementing preliminary feature extraction; S4, constructing a fault diagnosis model based on a cross-modal attention mechanism, using joint features to construct a training data set, and training a neural network model; S5, after the model training is completed, real-time fault diagnosis of the navigation device can be realized through data preprocessing technology and the fault diagnosis model based on the collected data.

2. The cross-modal attention fusion based navigation device real-time fault diagnosis method of claim 1, wherein, In the step S1, the data types are divided into four modes of sensor data, electrical data, acoustic data and text data, the sensor data includes GNSS position data, GNSS speed data, IMU acceleration data, IMU angular velocity data, temperature data and humidity data; the electrical data includes current, voltage and power data; the acoustic data includes device operation noise and ultrasonic data; the text data is the device operation log; the data collected at a fixed time interval constitutes a single sample, and the original sample set DataSet1 is constructed by collecting samples in normal state and various fault states.

3. The cross-modal attention fusion based navigation device real-time fault diagnosis method of claim 2, wherein, In the step S2, the three types of numerical mode data of sensor data, electrical data and acoustic data in each sample are screened for abnormal values by using the Z-score method, the missing values and abnormal values are filled by using linear interpolation, the data is smoothed by using the Lowess method, and finally the data is standardized by using the Min-Max normalization; the text type mode data is cleaned by using the special character removal, punctuation removal and pause word removal processing method; all the cleaned data together constitute the cleaned sample set DataSet2.

4. The cross-modal attention fusion based navigation device real-time fault diagnosis method of claim 3, wherein, In the step S3, in the sensor data, the GNSS position data includes a longitude coordinate sequence S1 and a latitude coordinate sequence S2, the GNSS speed data includes an x-axis speed sequence S3, a y-axis speed sequence S4 and a z-axis speed sequence S5, the IMU acceleration data includes an x-axis acceleration sequence S6, a y-axis acceleration sequence S7 and a z-axis acceleration sequence S8, the IMU angular velocity data includes an x-axis angular velocity time sequence S9, a y-axis angular velocity time sequence S10 and a z-axis angular velocity time sequence S11, the temperature data is a temperature value sequence S12, and the humidity data is a humidity percentage sequence S13, which are original data of 13 dimensions; the mean value and the standard deviation of S1 to S13 are extracted, the peak-to-peak value, the kurtosis value, the skewness value, the waveform factor and the pulse factor of S1 to S11 are extracted, and the average frequency, the center frequency and the frequency standard deviation of S1 to S11 are extracted after fast Fourier transform; A total of 114 dimensional features are extracted from the sensor data in each sample; the dimension is reduced by PCA principal component analysis, and 15 dimensional reconstructed features with the highest contribution rate are retained; In the electrical data, the current data is a direct current sequence E1, the voltage data is a direct voltage sequence E2, and the power data includes an active power sequence E3, a total of 3 dimensions of original data; 4 time domain features of average value, standard deviation, peak-valley difference, and change rate are extracted from S1, S2, and S3, and 3 frequency domain features of harmonic distortion rate, signal-to-noise ratio, and peak-to-average ratio are extracted after fast Fourier transform of S1, S2, and S3; A total of 21 dimensions of features are extracted from the electrical data in each sample; the dimension is reduced by PCA principal component analysis, and the 8 dimensions of reconstructed features with the highest contribution rate are retained; In the acoustic data, the running noise data is a wideband noise sequence A1, and the ultrasonic data is a high-frequency noise sequence A2, a total of 2 dimensions of original data; 3 time domain features of energy value, kurtosis value, and pulse factor are extracted from A1, 3 frequency domain features of mean frequency, center frequency, and frequency standard deviation are extracted after fast Fourier transform of A1, and 3 indexes of pulse count rate, pulse peak-to-peak value, and pulse average value are extracted after envelope demodulation analysis of A2, a total of 9 dimensions of features are extracted from the acoustic data in each sample; the dimension is reduced by PCA principal component analysis, and the 4 dimensions of reconstructed features with the highest contribution rate are retained; In the text data, 4 text statistical features of byte number mean value, byte number variance, maximum length of repeated logs, and log interval mean value are calculated for T1; After the above process, the DataSet2 sample is converted into a feature sample set DataSet3 containing the reconstructed features of each modality.

5. The cross-modal attention fusion based navigation device real-time fault diagnosis method of claim 4, wherein, In step S4, a fault diagnosis model is built, and the entire fault diagnosis model is built with an attention fusion mechanism as the core, and is composed of a feature isomorphism encoding layer, a modality calculation layer, a modality association layer, and a modality fusion layer to form a complete architecture.

6. The cross-modal attention fusion based navigation device real-time fault diagnosis method of claim 5, wherein, In the feature isomorphism encoding layer, a linear fully connected encoder maps the original heterogeneous modality features to a unified dimension space through four independent fully connected operations, specifically converting 15-dimensional sensor features, 8-dimensional electrical features, 4-dimensional acoustic features, and 4-dimensional text features into 64-dimensional isomorphic representations, establishing a standardized feature basis for subsequent processing.

7. The cross-modal attention fusion based navigation device real-time fault diagnosis method of claim 6, wherein, After entering the modality calculation layer, the system is processed in parallel in a target modality calculation path and a source modality calculation path; the target path generates a query matrix Q using a fully connected layer with an output of 128 dimensions, and the source path generates intermediate features through a fully connected layer with an output of 256 dimensions, and equally divides the intermediate features into a key matrix K and a value matrix V; after each sample is processed through this layer, four complete Q, K, and V matrix combinations are formed, corresponding to the four types of modalities.

8. The cross-modal attention fusion based navigation device real-time fault diagnosis method of claim 7, wherein, The modality association layer is the core interaction module of the model, and its multi-head attention mechanism drives the dynamic association calculation between modalities. This layer adopts a circular processing strategy, allowing each modality to take turns as the target modality while treating all modalities as source modalities, thereby forming sixteen combinations of four types of target modalities and four types of source modalities; Each combination generates an association sequence Attn with a length of 128 through the attention mechanism, fully capturing the cross-modality interaction features: The calculation formula of Attn is as follows: where Q is the target modality query matrix, K and V are the source modality key matrix and value matrix, d k is the scaling factor, and d k is uniformly 128 in this patent.

9. The cross-modal attention fusion based navigation device real-time fault diagnosis method of claim 8, wherein, The modal fusion layer adopts a gated neural network architecture to realize feature compression; sixteen groups of associated sequences are first integrated into a 16×128 associated matrix as input; a gated weight generation network learns the dynamic weight distribution between features through a fully connected layer and outputs a 16×16 weight matrix; the weight matrix and the original associated matrix are multiplied to realize weighted fusion, and the fusion result is compressed and expanded through an adaptive pooling layer to form a 64-dimensional one-dimensional sequence representing the essential features of the sample; This feature vector is directly connected to the fault classifier to establish a mapping relationship with the fault type through end-to-end training, and finally completes the fault diagnosis task.

Images