Multi-modal pulse fusion network prediction framework for aero-engine gas path performance parameters

Abstract

The invention provides a multi-modal pulse fusion network prediction framework for aero-engine gas path performance parameters, and belongs to the technical field of aero-engine gas path performance parameter prediction and fault diagnosis. The method comprises the following steps: firstly, acquiring flight data of an aero-engine, and preprocessing data of each sensor; secondly, constructing a multi-modal pulse fusion neural network model comprising a time sequence data pulse processing module, an image data pulse processing module, a multi-modal feature fusion module and a prediction output module; and finally, training the multi-modal pulse fusion neural network model, and predicting gas path performance parameters by using the trained model. According to the method, the leakage integration distribution spiking neuron and the attention mechanism are introduced, the depth feature extraction and fusion of the time sequence data set and the image data set are realized, the complex nonlinear correlation between the data is effectively captured, the accuracy of the prediction result is remarkably improved, and the real-time prediction requirement in the operation process of the aero-engine can be met.

Description

Technical Field

[0001] This invention belongs to the field of prediction and fault diagnosis technology of aero-engine gas path performance parameters. For the accurate prediction of key parameters of aero-engine gas path performance under complex operating conditions, it involves a model framework for predicting key parameters of aero-engine gas path performance using a multimodal pulse fusion neural network. Background Art

[0002] With the rapid development of aviation technology, the reliability and safety of aero engines, as the core components of aircraft, are of paramount importance. Engine gas path performance parameters, such as rotor speed, temperature, pressure, and flow rate, are key indicators for assessing their operating status, conducting fault diagnosis, and managing health. Accurately predicting these parameters is crucial for improving engine operational safety, optimizing maintenance plans, and achieving efficient health management. However, traditional gas path performance parameter prediction methods are mainly based on physical model theoretical analysis or empirical formulas. These methods are limited by the discrepancy between model assumptions and actual operating environments when dealing with complex conditions and variable operating environments, making it difficult to achieve high-precision predictions. This is especially true in scenarios with strong nonlinearity and multivariate coupling, where prediction accuracy and generalization ability are significantly insufficient. In recent years, gas path performance parameter prediction methods based on machine learning and deep learning have gradually become the mainstream research direction. Reference [1] = {Mei Y, Wu Y, Li L. Fault diagnosis and remaining useful life estimation of aero engine using LSTM neural network [C] / / IEEE International Conference on Aircraft UtilitySystems.IEEE, 2016.} The Long Short-Term Memory (LSTM) network was used to achieve good prediction results under complex operating conditions, multiple faults and strong noise. However, due to the limitation of memory unit capacity, the traditional LSTM network has inherent defects in handling long sequence dependencies and highly nonlinear mappings. Faced with the complex multivariate interaction mode in the prediction task of aero engine air path performance parameters, its modeling accuracy and dynamic adaptability are difficult to meet the actual needs. Reference [2] = {Zhou Heyang. Prediction of aero engine air path performance parameters based on dual-stream attention network [D]. Dalian University of Technology, 2022} A prediction model of aero engine air path performance parameters based on the Transformer algorithm was constructed, and the prediction accuracy was improved by adopting a dual-stream attention network structure on the basic Transformer model. However, the Transformer and its two-stream attention mechanism rely on continuous-valued matrix multiplication, and their computational complexity is quadratic with the sequence length. This results in long computation times in aero-engine aero-path performance parameter prediction tasks, making it difficult to meet real-time requirements. Although existing deep learning techniques have made some progress in aero-engine parameter prediction, the following problems still exist:

[0003] (1) Existing models based on traditional deep neural networks focus on the independent extraction of single modal features, which are insufficient for modeling complex nonlinear relationships between multi-source heterogeneous data and have low prediction accuracy for predicting the gas path performance parameters of aero-engines.

[0004] (2) Traditional deep learning methods generally rely on highly complex floating-point operations and large-scale matrix calculations, resulting in long model inference times. They are difficult to achieve rapid response in resource-constrained environments and cannot effectively support real-time prediction tasks during the operation of aero engines.

[0005] This invention proposes a novel multimodal pulse fusion network prediction framework model that can accurately predict and estimate key parameter values ​​of aero-engine gas path performance. Summary of the Invention

[0006] This invention addresses the limitations of existing technologies in multi-source data fusion and feature extraction by proposing a method for predicting aero-engine gas path performance parameters based on a multimodal spiking neural network. This method innovatively combines temporal pulse coding, image feature extraction, and an adaptive attention fusion mechanism to achieve deep feature mining of multi-source heterogeneous sensor data, significantly improving the accuracy, real-time performance, and dynamic adaptability to complex operating conditions in predicting gas path performance parameters.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A multimodal pulse fusion network prediction framework for aero-engine air path performance parameters is presented. This method, based on a multimodal pulse fusion neural network model, includes the following steps: First, aero-engine flight data is acquired, and preprocessing steps such as normalization, denoising, dimensionality reduction, and image format conversion are performed on the data from each sensor. Second, a multimodal pulse fusion neural network model is constructed, comprising a time-series data pulse processing module, an image data pulse processing module, a multimodal feature fusion module, and a prediction output module. Third, the multimodal pulse fusion neural network model is trained using the preprocessed aero-engine flight dataset. Finally, the trained multimodal pulse fusion neural network model is used to predict the aero-engine air path performance parameters. Details are as follows:

[0009] S1, Preprocessing of flight data for aircraft engines;

[0010] S1.1 Obtaining raw data:

[0011] Flight data is collected from multiple sensors on the aero-engine, including airflow performance parameters such as engine rotor speed, temperature, and gas flow rate. The measurement data from each sensor is in the form of a one-dimensional data sequence. The data collected by all sensors during a complete flight constitutes the raw data for subsequent model training and testing.

[0012] S1.2 Data normalization processing:

[0013] The original data obtained in S1.1 is processed using the min-maximum normalization method. This ensures that all input features have a similar scale, preventing certain features from dominating the model learning process.

[0014]

[0015] Where x represents the value of the original data; x′ represents the value of the original data after normalization, ranging from [0,1]; x min This represents the minimum value in the original data x; x max This represents the maximum value in the original data x.

[0016] S1.3, Denoise the raw data after processing in S1.2:

[0017] During the acquisition of aero-engine flight data, noise is inevitably present in the raw data due to factors such as sensor measurement accuracy, environmental interference, and signal transmission. This noise can increase data volatility, thereby interfering with the model's accurate understanding of the data and affecting prediction performance. Therefore, data denoising is an indispensable step in the data preprocessing stage, aiming to preserve the main characteristic trends of the data while effectively removing high-frequency interference and random fluctuations. This invention uses discrete wavelet transform to denoise the raw data processed in S1.2.

[0018] S1.4. Perform image format conversion on the raw data processed in S1.3:

[0019] First, given the lack of image modal information in the original dataset, this invention innovatively employs the Morlet wavelet transform algorithm to perform multi-scale decomposition and reconstruction of the temperature sensor time-series signal, generating a two-dimensional image feature matrix with spatiotemporal correlation. This process enhances the local dynamic feature representation capability through the oscillatory characteristics of the wavelet basis function, providing a high-information-entropy visual representation for subsequent cross-modal feature fusion. This process is implemented using Morlet wavelets in Matlab. Next, the temperature sensor data is removed from the original dataset. Finally, the temperature sensor data, after image format conversion, constitutes the image dataset for subsequent model training, denoted as x. image .

[0020] S1.5. Perform dimensionality reduction on the original data after processing in S1.4:

[0021] The correlation coefficient r between the performance parameter to be predicted in the original data and all other sensor parameters in the original data is calculated using the Pearson correlation coefficient, which is defined as Equation (3).

[0022]

[0023] Where, Xi and Y i These are the values ​​of the performance parameter to be predicted and the other sensor parameters, respectively. and These are their average values, and N is the length of the original data. When the absolute value of the correlation coefficient r between the sensor parameter and the performance parameter to be predicted is greater than the threshold δ, it is used as the input sensor for subsequent model training. Conversely, when the absolute value of the correlation coefficient r is less than the threshold δ (the threshold δ ranges from 0.6 to 0.9), the sensor parameter is considered to have a low correlation with the parameter to be predicted, and therefore is discarded. The original data after dimensionality reduction constitutes the time series dataset for subsequent model training, denoted as x. series .

[0024] Furthermore, the Pearson correlation coefficient r is a value used to measure the degree of linear correlation between a sensor data point and the performance parameter to be predicted. Its value ranges from [-1, 1], where a value of 1 indicates a perfectly positive correlation, a value of -1 indicates a perfectly negative correlation, and a value of 0 indicates no linear relationship between the two. The threshold δ is a screening criterion for dimensionality reduction, aiming to retain only sensor data that is sufficiently closely related to the predicted performance parameter, thereby achieving feature dimensionality reduction and model optimization.

[0025] S2. Construct a multimodal pulse fusion neural network model;

[0026] This invention introduces a leaky integrated firing spiking neuron model to sample pulses from image datasets and time-series datasets, outputting their corresponding pulse sequences. An attention mechanism is then used to effectively fuse multimodal information. This improvement allows the multimodal spiking fusion neural network model to comprehensively utilize multimodal data, more effectively mining its spatiotemporal features. Furthermore, the discretization of the pulse sequences enhances the prediction accuracy and real-time performance of the multimodal spiking fusion neural network model. Specifically, the multimodal spiking fusion neural network used in this invention comprises four modules: a time-series data pulse processing module, an image data pulse processing module, a multimodal feature fusion module, and a prediction output module. The multimodal spiking fusion neural network model is constructed using the preprocessed aero-engine flight data from step 1, as detailed below:

[0027] S2.1, Timing data pulse processing module;

[0028] The aforementioned time-series data pulse processing module will process the time-series dataset x series After leakage integration, the spiking neuron model is converted into spiking data, as shown in equation (4):

[0029] V(t)=V reset +(V(t)-V reset )·H(V thr-V(t))#(4)

[0030] Where V(t) is the membrane potential, V thr V is the pulse firing threshold potential. reset For resetting the potential, H(·) is the Herveside function, which triggers a discharge event when the membrane potential reaches a threshold. It is defined by equation (4).

[0031]

[0032] Where v represents the potential difference across the cell membrane.

[0033] With an input current, the membrane potential gradually increases and reaches the pulse firing threshold V. thr A "discharge" event is triggered at this time. After discharge, the membrane potential is immediately reset to V. reset Then it re-enters the integration phase. After pulse sampling, the time-series dataset will output a time-series pulse sequence, where each element is either 1 or 0, where 1 represents firing and 0 represents not firing. The output time-series pulse sequence is denoted as y. series ;

[0034] S2.2, Image Data Pulse Processing Module;

[0035] The image data pulse processing module first uses convolutional layers and pooling layers to process the image dataset x. image Preliminary feature extraction is performed, and then the obtained features are input into a fully connected layer for further feature extraction. Finally, pulse sampling is achieved through the leaky integrated firing spiking neuron model introduced in Equation (4), and the image dataset is output as an image pulse sequence. The output image pulse sequence is denoted as y. image ;

[0036] S2.3, Multimodal Feature Fusion Module;

[0037] The multimodal feature fusion module will process the output y from the time-series data pulse processing module. series and the output of the image data pulse processing module y image Feature fusion is performed using a soft attention mechanism.

[0038] First, define two learnable parameters w. series and w img They are then standardized using the softmax function to ensure that their sum is 1. The process is shown in equations (6)-(7).

[0039]

[0040] Where, α series This indicates the output y from the timing data pulse processing module. seriesThe corresponding feature weight coefficients; w series α series The corresponding attention learnable weight parameters; α img This indicates the output y from the image data pulse processing module. imge The corresponding feature weight coefficients; w img α img The corresponding attention can learn weight parameters; exp(.) represents the exponential function operation;

[0041] Next, output y of the timing pulse. series and image pulse output y image Perform weighted fusion to obtain the fusion feature y fusion :

[0042] y fusion =α series ·y series +α img ·y image #(8)

[0043] S2.4, Prediction Output Module;

[0044] The prediction output module will fuse the feature vector y from S2.3. fusion The data is fed into a fully connected layer for further processing. This fully connected layer uses a linear activation function to output continuous predictions of gas path performance parameters. Its expression is:

[0045]

[0046] Among them, W out and b out These are the weights and biases of the fully connected layer. This is the final prediction result.

[0047] S3. Training and optimization of multimodal pulse fusion neural network model.

[0048] In the training phase of the multimodal pulse fusion neural network model, a parameter update method based on backpropagation is adopted, and the learnable weights in the network are gradually optimized and adjusted in combination with the Adam optimization algorithm. First, the aero-engine flight dataset is divided into training set and test set according to different flight cycles. The training set is used for model parameter learning, and the test set is used for model performance evaluation and verification. Second, before the training starts, all learnable weight parameters in the network are randomly initialized. In each round of training, training samples are input, the model prediction output is calculated, the prediction error is calculated according to the loss function, the gradient is calculated through the backpropagation algorithm, and the network weight parameters are updated using the Adam optimization algorithm. After completing one weight update, the next round of iteration begins. Third, the training process continues until either of the following conditions is met and then stops: (1) The preset maximum number of training rounds (epochs) is reached. (2) The loss function value on the validation set changes less than the preset threshold θ (0.0001 to 0.001) within several consecutive rounds, which is considered convergence. Finally, after training, the predictive performance of the model is evaluated using a test set, and the weights of the best-performing model are saved for subsequent prediction tasks of aero-engine gas path performance parameters. The specific process is as follows:

[0049] S3.1 Loss Function Design;

[0050] To measure the deviation between the output of the multimodal pulse fusion neural network model and the actual gas path performance parameters, a suitable loss function needs to be constructed as the objective function during the training process. Given that the task of this invention is of the regression type (such as predicting continuous values ​​of parameters like speed and pressure), the Mean Squared Error (MSE) loss function is chosen as the performance metric. Its expression is shown below:

[0051]

[0052] Among them, y i This represents the true value of the i-th sample. It represents the predicted output of the corresponding multimodal pulse fusion neural network model, where N is the number of samples.

[0053] The MSE function effectively drives the optimization algorithm to search for a better solution in the parameter space by penalizing the squared error between the predicted and actual values. During training, this loss function serves as the basis for backpropagation, guiding the model to continuously adjust its internal weights, thereby enhancing its ability to fit the changing trends of aero-engine performance and improving prediction accuracy.

[0054] S3.2 Optimization algorithm selection;

[0055] The appropriate selection of the optimizer has a crucial impact on the convergence performance and training efficiency of deep learning models. This invention employs an improved optimization method integrating Adaptive Gradient (AdaGrad) and Root Mean Square Propagation (RMSProp) mechanisms, using the Adam (Adaptive Moment Estimation) optimizer. This method dynamically adjusts the learning rate of each parameter by comprehensively estimating the first and second moments of the current gradient of the loss function with respect to the model parameters, thereby improving the model's adaptability to different data distributions. Its core calculation steps are as follows:

[0056] First-order moment estimate (momentum term):

[0057] m t =β1m t-1 +(1-β1)g t #(11)

[0058] Second-order moment estimation (exponential weighted average of squared gradients):

[0059]

[0060] Deviation correction (eliminating the influence of initial offset):

[0061]

[0062] Parameter update formula:

[0063]

[0064] Among them, g t Let m represent the gradient at the current time t. t and v t Let α represent the first-order moment estimate and the second-order moment estimate at time t, respectively, where α is the initial learning rate, β1 and β2 are the exponential decay rates of the first-order moment estimate and the second-order moment estimate, respectively, and ∈ is a small constant to prevent the denominator from being zero; m t-1 v represents the first moment estimate at time t-1; t-1 This represents the second moment estimate at time t-1; Indicates the relationship between m t The estimated value after bias correction; Indicates to v t The estimated value after bias correction; This represents β1 raised to the power of t; θ represents β² raised to the power of t; t θ represents the parameters updated at time t. t-1 This represents the parameters before the update at time t-1.

[0065] This optimizer features self-adjusting learning rate, adaptability to sparse gradients, and faster convergence, making it particularly suitable for time-series data modeling tasks. In this invention, Adam effectively improves the convergence stability and prediction performance of multimodal pulse fusion neural network models under high-dimensional multimodal inputs.

[0066] S4. Output the prediction results to achieve accurate prediction of the gas path performance parameters of aero-engines.

[0067] The trained multimodal pulse fusion neural network model is applied to flight data input and outputs the corresponding aero-engine air path performance parameter values ​​to achieve dynamic evaluation and trend prediction of engine operating status.

[0068] The root mean squared error (RMSE) and mean absolute error (MAE) are selected as the evaluation indicators for the prediction results.

[0069]

[0070] Where yi represents the true value. RMSE and MAE represent the predicted values, and N is the total number of samples. The smaller the values ​​of RMSE and MAE, the closer the predicted values ​​are to the actual values, and the better the prediction performance.

[0071] The beneficial effects of this invention are as follows:

[0072] This invention constructs a multimodal pulse fusion network prediction framework suitable for complex conditions by performing a comprehensive preprocessing process on aero-engine flight data, extracting pulse sequences using a spiking neural network, and dynamically weighting and fusing the extracted pulse sequences from multimodal data using an attention mechanism. Compared with traditional methods, it has the following significant advantages:

[0073] (1) Significantly Improved Prediction Accuracy: The multimodal spiking neural network architecture constructed in this invention achieves dynamic sparse encoding of temporal features through leaky integration of firing spiking neurons, and adaptively adjusts the weights of multimodal information using a soft attention mechanism, forming a deep modeling capability of the inherent nonlinear mapping relationship of the data. In extreme scenario tests such as complex working condition switching and sudden fault injection, compared with traditional methods, it demonstrates the ability to capture weak feature signals beyond the threshold, significantly improving the accuracy of prediction results and fully verifying the robustness advantage of the model under non-ideal conditions.

[0074] (2) It has real-time prediction capability: The multimodal pulse fusion neural network model uses discrete pulse sequences for network training, which can improve the training speed of the model and meet the real-time prediction requirements during the operation of aero-engines. Attached Figure Description

[0075] Figure 1 This invention presents the overall structure of a multimodal pulse fusion neural network model for predicting aero-engine gas path performance parameters.

[0076] Figure 2 shows a flight dataset provided by the Aero Engine Research Institute; Figure 2(a) shows the raw data of parameters D8, T1, a1, and a2; Figure 2(b) shows the raw data of parameters T6, H, M, and V; Figure 2(c) shows the raw data of parameters n2, n1, and PLA.

[0077] Figure 3 The image shows the effect of wavelet transform denoising.

[0078] Figure 4 This is a schematic diagram illustrating the composition of multimodal data in this invention.

[0079] Figure 5 The figure shows the prediction results of the multimodal pulse fusion network model proposed in this invention for the low-pressure rotor speed N1.

[0080] Figure 6 This is a comparison of the number of convergence cycles during training between the multimodal pulse fusion network proposed in this invention and the three contrasting neural networks. Detailed Implementation

[0081] To better understand the technical solution of the present invention, the method of the present invention will be described in detail below with reference to specific embodiments and accompanying drawings.

[0082] This implementation presents a multimodal pulse fusion neural network prediction model for aero-engine gas path performance parameters, with appended... Figure 1 The overall structure of the model is represented by the following steps:

[0083] S1, Flight Data Preprocessing for Aircraft Engines

[0084] S1.1 Obtaining raw data:

[0085] To illustrate the specific implementation details of this invention in more detail, it is based on an aero-engine flight dataset collected by Dalian University of Technology in conjunction with a domestic aero-engine research institute. This flight data is divided into two groups, representing complete flight data collected from two flight missions performed by the same aircraft. Each group contains data from 11 sensors, as shown in Table 1, with approximately 100,000 data points per group. The variation curves of each sensor are shown in Figure 2.

[0086] Table 1 Dataset Parameter Table

[0087]

[0088] S1.2 Data normalization processing:

[0089] The original data obtained from S1.1 is processed using the minimum-maximum normalization method of equation (1). This ensures that all input features have similar scales and avoids certain features dominating the model learning process.

[0090] S1.3, Denoise the raw data after processing in S1.2:

[0091] During the acquisition of aero-engine flight data, noise is inevitably present in the raw data due to factors such as sensor measurement accuracy, environmental interference, and signal transmission. This noise can increase data volatility, thereby interfering with the model's accurate understanding of the data and affecting prediction performance. Therefore, denoising is an indispensable step in the data preprocessing stage, aiming to preserve the main characteristic trends of the data while effectively removing high-frequency interference and random fluctuations. In this implementation, discrete wavelet transform is used to denoise the raw data processed by S1.2. This process uses a wavelet denoising method combining the Daubechies wavelet basis function with the Garrote threshold function. Figure 2 shows the normalized T1 denoising effect.

[0092] S1.4. Perform image format conversion on the raw data processed in S1.3:

[0093] First, addressing the issue of the lack of image-formatted data in the original dataset, the T6 temperature sensor data was converted into two-dimensional image data using wavelet transform. This process was implemented in Matlab using Morlet wavelets. Ten data points were input at a time, generating an image of 28×28 pixels with a step size of 1. Next, the temperature sensor data was removed from the original dataset. Finally, the image dataset for subsequent model training, denoted as x, was constructed from the converted temperature sensor data. image The process is as follows (attached). Figure 4 As shown.

[0094] S1.5. Perform dimensionality reduction on the original data after processing in S1.4:

[0095] In this invention, the performance parameter to be predicted is selected as N1. The Pearson correlation coefficient between the remaining sensor data and N1 is calculated using equation (3). The threshold δ is set to 0.75. After calculation, the sensors with high correlation are D8, T6, a2, M, V, N2, and PLA. The remaining sensors are removed. The original data after dimensionality reduction constitutes the time series dataset for subsequent model training, denoted as x. series In this embodiment, the threshold δ is in the range of 0.75.

[0096] S2. Construct a multimodal pulse fusion neural network model;

[0097] This implementation first applies an image dataset x by introducing a leaky integrated firing spiking neuron model. image With time series dataset x series Pulse sampling is performed separately, and the corresponding pulse sequences are then output. Finally, an attention mechanism is used to effectively fuse multimodal information. This improvement allows the multimodal pulse fusion neural network model to comprehensively utilize multimodal data, more effectively extract its spatiotemporal features, and improves the prediction accuracy and real-time performance of the model through the discretization of pulse sequences. Specifically, the multimodal pulse fusion neural network used in this invention includes the following four modules: a time-series data pulse processing module, an image data pulse processing module, a multimodal feature fusion module, and a prediction output module. The multimodal pulse fusion neural network model is constructed using the preprocessed aero-engine flight data from step 1, as detailed below:

[0098] S2.1, Timing data pulse processing module;

[0099] The aforementioned time-series data pulse processing module will process the time-series dataset x series After leakage integration, the spiking neuron model is converted into spiking data, as shown in equation (4): Set the firing threshold V thr It is 0.85, V reset The output timing pulse signal is denoted as y, which is 0. series ;

[0100] S2.2, Image Data Pulse Processing Module;

[0101] The image data pulse processing module first uses convolutional layers and pooling layers to process the image dataset x. image Preliminary feature extraction is performed, and then the obtained features are input into a fully connected layer for further feature extraction. Finally, pulse sampling is achieved through the leaky integrated firing spiking neuron model introduced in equation (4), and the firing threshold V is set. thr It is 0.85, V reset The value is 0. The image dataset will output an image pulse sequence. The output pulse sequence is denoted as y. image ;

[0102] S2.3, Multimodal Feature Fusion Module;

[0103] The multimodal feature fusion module will process the output y from the time-series data pulse processing module. series and the output of the image data pulse processing module y image Feature fusion is performed using a soft attention mechanism. The fused features y are calculated according to equations (6)-(8). fusion The initialization of w is as follows. series and w imgBoth are 0.5.

[0104] S2.4, Prediction Output Module;

[0105] The prediction output module will fuse the feature vector y from S2.3. fusion The input is fed into a fully connected layer for further processing. This fully connected layer employs a linear activation function, designed to output continuous gas path performance parameter predictions.

[0106] S3, Model Training and Optimization;

[0107] The first flight data processed in step S1 is used as the training set and input into the network. All learnable parameters in the network are learned through the loss function and optimization algorithms (11)-(14). The initial learning rate α is set to 0.0001; the exponential decay rates β1 and β2 of the first moment estimation and the second moment estimation are 0.9 and 0.999, respectively; the small constant ∈ to prevent the denominator from being zero is 0.00001; the maximum number of training epochs is 30; and the preset threshold θ of the loss function in this implementation is 0.0002.

[0108] S4. Prediction results generation:

[0109] In this implementation, the data from the second flight was selected as the test dataset. The model trained in step S2 was validated on the second flight dataset processed in step S1. Specifically, the trained Multimodal Spike Neural Network (MMSNN) model was compared with the traditional Single-Mode Spike Neural Network (SNN) model and two mainstream time series prediction algorithms: Gated Recurrent Unit (GRU) and Transformer. The comparison images of the MMSNN prediction results and the actual values ​​are attached. Figure 5 As shown, the specific values ​​of RMSE and MAE between the prediction results of each model and the actual value are calculated according to equations (15)-(16) as evaluation indicators of the prediction accuracy of each model. The smaller the value, the closer the prediction result is to the actual value and the better the model prediction result. The specific values ​​are shown in Table 2.

[0110] Table 2 Model Prediction Results Error

[0111] Model RMSE MAE MMSNN 0.041024 0.026815 SNN 0.055889 0.039281 GRU 0.050131 0.035134 Transformer 0.044191 0.030758

[0112] As can be seen, the multimodal spiking fusion network proposed in this invention achieves the best performance across all evaluation metrics, with a root mean square error (RMSE) of 0.041024 and a mean absolute error (MAE) of 0.026815. The performance of the gated recurrent unit (GRU) model and the Transformer model is inferior to that of the proposed method. The traditional spiking neural network exhibits poor prediction performance, with an RMS of 0.055889 and an MAE of 0.039281. Through the effective fusion of multimodal features, the multimodal spiking fusion network model proposed in this invention demonstrates significant advantages in capturing nonlinear dynamics and detailed features in data, significantly overcoming the shortcomings of traditional single-modal models in feature extraction and complex data fitting.

[0113] The number of convergence cycles for each model during training is shown in the appendix. Figure 6 As shown, the multimodal pulse fusion network prediction model proposed in this invention converges in about 25 training cycles, which is faster than the other comparative models and improves the real-time performance of aero-engine gas path performance parameter prediction.

[0114] This invention improves the perception of local dynamic changes by introducing wavelet transform to convert temperature-related time-series signals into two-dimensional images. It utilizes leaky integrated firing spiking neurons to extract pulse sequences from time-series and image datasets, and achieves adaptive fusion of multimodal pulse features through a soft attention mechanism, enhancing the collaborative modeling capability of multi-source information. By combining the mean square error loss function and the Adam optimization algorithm, it effectively improves the accuracy and real-time performance of predicting aero-engine gas path performance parameters, demonstrating superior performance in both convergence speed and prediction accuracy compared to traditional methods.

[0115] The embodiments described above are merely illustrative of the implementation methods of the present invention, but should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the protection scope of the present invention.

Claims

1. A multimodal pulse fusion network prediction framework for aero-engine gas path performance parameters, characterized in that, The multimodal pulse fusion network prediction framework is a method for predicting aero-engine gas path performance parameters based on a multimodal pulse fusion neural network model, including the following steps: Step S1: Obtain the raw flight data of the aero-engine and preprocess the data from each sensor. The preprocessing includes normalization, noise reduction, image format conversion and dimensionality reduction to obtain the aero-engine flight dataset. Step S2: Construct a multimodal pulse fusion neural network model that includes a time-series data pulse processing module, an image data pulse processing module, a multimodal feature fusion module, and a prediction output module; Step S3: Using the preprocessed aero-engine flight dataset, train and optimize the multimodal pulse fusion neural network model to obtain a trained multimodal pulse fusion neural network model; During the training phase of the multimodal pulse fusion neural network model, a parameter update method based on backpropagation mechanism is adopted, and the learnable weights are gradually optimized and adjusted in combination with the Adam optimization algorithm. Step S4: Use the trained multimodal pulse fusion neural network model to predict the gas path performance parameters of the aero-engine.

2. The multimodal pulse fusion network prediction framework for aero-engine gas path performance parameters according to claim 1, characterized in that, The specific steps of S1 are as follows: S1.1 Obtaining raw data: Flight data is collected from multiple sets of sensors on the aircraft engine. The measurement data of each set of sensors is in the form of a one-dimensional data sequence. The data collected by all sensors during a complete flight constitutes the raw data. S1.2 Data normalization processing: The original data obtained from S1.1 is processed using the minimum-maximum normalization method; S1.

3. Use discrete wavelet transform to denoise the original data after processing in S1.2: S1.

4. Perform image format conversion on the raw data processed in S1.3: First, the temperature sensor data in the original dataset is converted into two-dimensional image data using wavelet transform. Then, the temperature sensor data is removed from the original dataset. Finally, the converted temperature sensor data forms an image dataset, denoted as x. image ; S1.

5. Perform dimensionality reduction on the data processed in S1.4: The correlation coefficient r between the performance parameter to be predicted in the data and all other sensor parameters in the original data is calculated using the Pearson correlation coefficient. When the absolute value of the correlation coefficient r is greater than the threshold δ, it is used as the input sensor for subsequent model training. Conversely, when the absolute value of the correlation coefficient r is less than the threshold δ, the sensor parameter is considered to have low correlation with the parameter to be predicted and is removed. The data after dimensionality reduction constitutes a time-series dataset, which serves as an aero-engine flight dataset, denoted as x. series .

3. The multimodal pulse fusion network prediction framework for aero-engine gas path performance parameters according to claim 2, characterized in that, In S1.5, the correlation coefficient r is defined as Equation (3); Where, X i and Y i These are the values ​​of the performance parameter to be predicted and the other sensor parameters, respectively. and These are their average values, and N is the length of the original data; the correlation coefficient r ranges from [-1, 1].

4. The multimodal pulse fusion network prediction framework for aero-engine gas path performance parameters according to claim 2, characterized in that, In step S1.5, the threshold δ ranges from 0.6 to 0.

9.

5. The multimodal pulse fusion network prediction framework for aero-engine gas path performance parameters according to claim 2, characterized in that, Step S2 is as follows: S2.1, Timing data pulse processing module; The aforementioned time-series data pulse processing module will process the time-series dataset x series After leakage integration, the spiking neuron model is converted into spiking data, as shown in equation (4): V(t)=V reset +(V(t)-V reset )·H(V thr -V(t))(4) Where V(t) is the membrane potential, V thr V is the pulse firing threshold potential. reset For resetting the potential, H(·) is the Herveside function, which triggers a discharge event when the membrane potential reaches a threshold. It is defined by equation (5). Where v represents the potential difference; After pulse sampling, the time-series dataset will output a time-series pulse sequence, denoted as y. series ; S2.2, Image Data Pulse Processing Module; The image data pulse processing module first uses convolutional layers and pooling layers to process the image dataset x. image Preliminary feature extraction is performed, followed by inputting the obtained features into a fully connected layer for further feature extraction. Finally, a leaky ensemble firing spiking neuron model is used to implement pulse sampling, and the image dataset will be output as an image pulse sequence; the output pulse sequence is denoted as y. image ; S2.3, Multimodal Feature Fusion Module; The multimodal feature fusion module will process the output y from the time-series data pulse processing module. series and the output of the image data pulse processing module y image A soft attention mechanism is used for feature fusion to obtain the fused feature vector y. fusion ; S2.4, Prediction Output Module; The prediction output module will fuse the feature vector y from S2.

3. fusion The data is input to the fully connected layer for processing, and outputs continuous predicted values ​​of gas path performance parameters.

6. The multimodal pulse fusion network prediction framework for aero-engine gas path performance parameters according to claim 5, characterized in that, In step S2: Specifically, S2.3 is as follows: Define two learnable parameters w series and w img And use the softmax function to standardize it; the process is as shown in equations (6)-(7); Where, α series This indicates the output y from the timing data pulse processing module. series The corresponding feature weight coefficients; w series α series The corresponding attention learnable weight parameters; α img This indicates the output y from the image data pulse processing module. imge The corresponding feature weight coefficients; w img α img The corresponding attention can learn weight parameters; exp(.) represents the exponential function operation; Output y of timing pulse series and image pulse output y image Perform weighted fusion to obtain the fusion feature y fusion : and fusion =α series ·and series +α img ·and image (8) The fully connected layer in S2.4 uses a linear activation function, the expression of which is: Among them, W out and b out These are the weights and biases of the fully connected layer. This is the final prediction result.

7. The multimodal pulse fusion network prediction framework for aero-engine gas path performance parameters according to claim 5, characterized in that, Step S3 is as follows: First, the aero-engine flight dataset is divided into training and testing sets according to different flight cycles; the training set is used for model parameter learning, and the testing set is used for model performance evaluation and validation. Secondly, before training begins, all learnable weight parameters in the network are randomly initialized; in each training round, training samples are input, the model's predicted output is calculated, the prediction error is calculated based on the loss function, the gradient is calculated using the backpropagation algorithm, and the network weight parameters are updated using the Adam optimization algorithm; after completing one weight update, the next iteration begins; thirdly, the training process continues until the termination condition is met. Finally, after training is completed, the predictive performance of the model is evaluated using a test set, and the weights of the model with the best performance are saved to obtain the trained multimodal spiking fusion neural network model.

8. The multimodal pulse fusion network prediction framework for aero-engine gas path performance parameters according to claim 7, characterized in that, The termination condition is that the process will stop if any of the following conditions are met: (1) Reach the preset maximum number of training epochs; (2) If the loss function value on the validation set changes less than a preset threshold θ in several consecutive rounds, it is determined to be converged; the preset threshold θ ranges from 0.0001 to 0.

001.

9. The multimodal pulse fusion network prediction framework for aero-engine gas path performance parameters according to claim 7, characterized in that, Step S3 is as follows: S3.1 Loss Function Design; The mean squared error loss function (MSE) was selected as the performance metric. S3.2 Optimization algorithm selection; An improved optimization method integrating adaptive gradient and root mean square propagation mechanisms is adopted, using the Adam optimizer. By comprehensively estimating the first and second moments of the current gradient, the learning rate of each parameter is dynamically adjusted to improve the model's adaptability under different data distributions. The core calculation steps are as follows: First moment estimate, momentum term: m t =β1m t-1 +(1-β1)g t (11) Second-order moment estimation, exponential weighting of the squared gradient: Deviation correction eliminates the influence of initial offset: Parameter update formula: Among them, g t Let m represent the gradient at the current time t. t and v t Let α represent the first-order moment estimate and the second-order moment estimate at time t, respectively, where α is the initial learning rate, β1 and β2 are the exponential decay rates of the first-order moment estimate and the second-order moment estimate, respectively, and ∈ is a constant to prevent the denominator from being zero; m t-1 v represents the first moment estimate at time t-1; t-1 This represents the second moment estimate at time t-1; Indicates the relationship between m t The estimated value after bias correction; Indicates to v t The estimated value after bias correction; This represents β1 raised to the power of t; θ represents β² raised to the power of t; t θ represents the parameters updated at time t. t-1 This represents the parameters before the update at time t-1.

10. The multimodal pulse fusion network prediction framework for aero-engine gas path performance parameters according to claim 7, characterized in that, Step S4 specifically involves: The trained multimodal pulse fusion neural network model is applied to flight data input and outputs the corresponding aero-engine air path performance parameter values ​​to achieve dynamic evaluation and trend prediction of engine operating status. The root mean square error (RMSE) and mean absolute error (MAE) were selected as the evaluation indicators for the prediction results.

Images