A device life prediction method based on a dual-channel lightweight convolutional Transformer model

Abstract

The application provides a device life prediction method based on a dual-channel lightweight convolutional Transformer model, and steps are as follows: monitoring data of a key device in an existing data set is acquired; the monitoring data is preprocessed, and a data set is constructed; the data set is divided into a training set and a test set; a DLCformer model is constructed, which comprises a position coding layer, a dual-channel lightweight ConvFormer mechanism, a feature fusion layer and a full connection layer connected in sequence; the dual-channel lightweight ConvFormer mechanism comprises a lightweight time sequence encoder and a lightweight feature encoder connected in parallel; training data in the training set is input into the DLCformer model for training, so that a trained DLCformer model is obtained; test data in the test set is input into the trained DLCformer model, so that a predicted device life is obtained. The application adopts a dual-channel parallel structure, encodes features in time and sensor dimensions respectively, enhances the space-time feature capturing capability in the degradation process, and has significant advantages in prediction accuracy and calculation efficiency, and has good engineering practicability.

Description

Technical Field

[0001] This invention relates to the technical field of equipment life prediction, and in particular to a method for predicting equipment life based on a dual-channel lightweight convolutional Transformer model. Background Technology

[0002] Prognostics and Health Management (PHM) technology for critical equipment is a core technology for ensuring reliable system operation and improving equipment operating efficiency, and it has broad application prospects in key fields such as industrial electronics, aerospace, and new energy. In recent years, degradation data-driven methods based on artificial intelligence, especially deep learning, have become a research hotspot for predicting remaining useful life (RUL) and assessing equipment health status.

[0003] Therefore, using multi-sensor monitoring data to predict the remaining service life (RUL) of equipment is a core task of industrial equipment health management. This type of monitoring data not only reflects the dynamic patterns and temporal dependencies of equipment operating status, but also implies the inherent coupling and mutual influence relationships between different sensor channels. Effective capture and deep fusion of these two types of information are crucial for improving the accuracy and robustness of RUL predictions. While the standard Transformer model and its variants have demonstrated good capabilities in modeling long-range time dependencies, their self-attention mechanism primarily models in the time dimension, often neglecting the potentially important spatial correlations between sensors within the same time step, making it difficult to comprehensively represent complex spatiotemporal interaction structures.

[0004] To address the aforementioned issues, existing research has proposed a dual-channel modeling strategy. This strategy uses parallel network structures to model information in both temporal and spatial dimensions, enabling collaborative modeling of multidimensional degradation processes. While such methods have achieved some performance breakthroughs, current dual-channel architectures combined with Transformer models face two main challenges: First, traditional self-attention mechanisms tend to focus on global feature modeling, limiting their ability to characterize key local details such as early degradation features and transient mutations, which can easily lead to the loss of local patterns. Second, the computational complexity of this mechanism is quadratic with the length of the input sequence or the number of sensor channels. Especially in dual-channel modeling, the overlapping of temporal and spatial encoding paths significantly increases computational burden and memory requirements, limiting its deployment feasibility in resource-constrained scenarios.

[0005] For example, invention patent CN113486578A proposes a method for predicting the remaining life of equipment in industrial processes. This invention includes preprocessing raw sensor data, performing time window processing on the preprocessed data, setting remaining life labels, constructing a temporal convolutional neural network model based on a channel attention mechanism, training the constructed temporal convolutional neural network model using training data, and using the trained temporal convolutional neural network model to predict remaining life. This invention uses a temporal convolutional neural network as the main feature learning model, which has the advantage of parallel computing and reduces the number of parameters in the model. It also utilizes a channel attention mechanism to extract important fault degradation information, automatically extracting information in each channel and distinguishing the importance of the information. However, this invention has the following problems: reducing the number of parameters in the model may lead to insufficient capture of complex degradation patterns, failure of multi-sensor fusion, and weakens the model's generalization ability to predict remaining life across different devices. Summary of the Invention

[0006] To address the technical problem of existing equipment lifetime prediction methods neglecting the potential important spatial correlations between sensors at the same time step and failing to fully represent the complex spatiotemporal feature interaction structure, this invention proposes an equipment lifetime prediction method based on a dual-channel lightweight convolutional Transformer (DLCformer) model. A lightweight convolutional self-attention (LCSA) mechanism is designed, which enhances the local feature extraction capability by introducing convolution operations and effectively reduces the computational complexity of self-attention by employing a broadcasting operation. Then, a dual-channel parallel structure is used to encode features in both time and sensor dimensions, enhancing the ability to capture spatiotemporal features of the degradation process, improving the prediction accuracy of remaining equipment lifetime while maintaining the algorithm's computational efficiency.

[0007] To achieve the above objectives, the technical solution of this invention is as follows: a device lifetime prediction method based on a dual-channel lightweight convolutional Transformer model, comprising the following steps:

[0008] Step 1: Obtain operational status monitoring data for existing key equipment;

[0009] Step 2: Preprocess the operational status monitoring data; and construct a dataset, dividing the dataset into a training set and a test set;

[0010] Step 3: Construct the DLCformer model: It includes a positional encoding layer, a dual-channel lightweight ConvFormer mechanism, a feature fusion layer, and a fully connected layer connected in sequence; the dual-channel lightweight ConvFormer mechanism includes a lightweight temporal encoder and a lightweight feature encoder connected in parallel;

[0011] Step 4: Input the training data from the training set into the DLCformer model for training to obtain the trained DLCformer model; input the test data from the test set into the trained DLCformer model to obtain the predicted device lifespan.

[0012] Preferably, the position encoding layer is provided with an embedding layer at the front end. The embedding layer performs feature dimension transformation and spatial mapping on the input sequence obtained after preprocessing. The position encoding layer embeds spatiotemporal position information into the time series after feature dimension transformation and spatial mapping.

[0013] Preferably, the method for data processing in the location coding layer is as follows:

[0014] X E =X′+X PE

[0015] in, The input data for the input position encoding layer is given by N, where N is the input feature dimension and T is the total length of the sequence; X PE This is a position coding matrix based on the frequency domain characteristics of sine and cosine functions;

[0016] The location encoding matrix is

[0017]

[0018] In the formula, i1 is the index of the dimension pair, and D model For model feature dimensions, 2i1 / D model is the frequency of the i1th dimension pair; pos is the position index of the corresponding sequence.

[0019] Preferably, the lightweight temporal encoder models the temporal dependencies within the data of the input sequence; the lightweight feature encoder processes the feature correlations between different sensors within each time step.

[0020] The lightweight temporal encoder includes multiple lightweight temporal attention layers connected in sequence, and the lightweight feature encoder includes multiple lightweight feature attention layers connected in sequence; the lightweight temporal attention layers and the lightweight feature attention layers are implemented using a lightweight convolutional self-attention mechanism, which includes convolution and self-attention mechanisms connected in sequence.

[0021] Preferably, the lightweight temporal attention layer of the lightweight temporal encoder is implemented as follows:

[0022] Replace the standard linear projection operation with convolution operation:

[0023]

[0024] In the formula, This is the input matrix after processing by the position coding layer. Let K be the time score vector, K be the key matrix, V be the value matrix, SoftMax() be the normalization exponential function, ReLU(·) be the activation function, and * be the convolution operation. All are convolution weight matrices with a kernel size of 1; the convolution weight matrix W τ Used to fit the input matrix X E The feature distribution is calculated and the importance of each time step is output. After normalization by the SoftMax function, the time score vector τ is obtained.

[0025] The self-attention mechanism achieves implicit dimensionality expansion through tensor broadcasting, using a time-based scoring vector. After performing element-wise broadcast multiplication with the key matrix K, summation is performed along the time axis to obtain the local feature weight γ;

[0026] After the local feature weights γ and the value matrix V are fused through a broadcast operation, they are linearly transformed through a convolutional layer with a kernel size of 1 to obtain the final output representation Z.

[0027] Preferably, the lightweight feature attention layer of the lightweight feature encoder is implemented as follows:

[0028] The input matrix X is subjected to a permutation operation. E The input matrix X is obtained by permuting the dimensions to obtain the fusion of feature information and temporal location information. E T ;

[0029] Feature vectors are generated through convolution operations. The convolution operation between the key matrix K′ and the value matrix V′ is as follows:

[0030]

[0031] In the formula, For a trainable parameter matrix, The generated feature vectors are K′, the key matrix, and V′, the value matrix.

[0032] The feature vector τ′ generated by the tensor broadcasting mechanism is multiplied element-wise with the key matrix K′ and then summed along the nodes to obtain the feature weight γ′. The value matrix V′ is dynamically modulated by the feature weight γ′, and combined with convolution operation to obtain the local fusion feature X′ of cross-sensor features. T .

[0033] Preferably, the local feature weights are:

[0034] In the formula, T is the total length of the sequence, and i is the index of the sequence time step. For broadcast operation, τ i K is the i-th element in the time-scoring vector τ. i It is the i-th row in the key matrix K.

[0035] The final output representation Z is

[0036] In the formula, V is the convolution weight matrix. j γ represents the feature information related to the j-th node in the value matrix V. j The weight value corresponding to the j-th node in the local feature weight γ;

[0037] The input matrix Permute() is a function that rearranges the dimensions of a tensor.

[0038] The feature weights The local fusion features

[0039] In the formula, W T X Let K be the convolution weight matrix. j ' represents the eigenvector of the key matrix K' at the j-th node; τ j ' represents the element at the j-th node of the generated feature vector; γ i ' represents the weight value of feature weight γ' at the i-th position; V i ' represents the eigenvector of the value matrix V' at the i-th position.

[0040] Preferably, both the lightweight temporal encoder and the lightweight feature encoder further include a first normalization layer, a multi-layer feedforward network, and a second normalization layer connected in sequence. The first normalization layer is connected to the output of the last lightweight temporal attention layer of the lightweight temporal encoder or the output of the last lightweight feature attention layer of the lightweight feature encoder. The input matrix processed by the position encoding layer is connected to the residual of the input of the first normalization layer, and the output matrix of the first normalization layer is connected to the input of the second normalization layer.

[0041] The calculation process of the feedforward network is as follows: it receives the output of the lightweight temporal attention layer or the lightweight feature attention layer and the features after the first addition and normalization process, performs nonlinear enhancement, and then the output is normalized again by the second addition and normalization to form the final output of the entire lightweight temporal encoder or lightweight feature encoder.

[0042] Preferably, the feature fusion layer uses a combination of splicing operations and linear transformations to aggregate the final outputs of the lightweight feature encoder and the lightweight temporal encoder, resulting in a fused feature F. d =Concat(F S ,F T W;

[0043] In the formula, F S For the final output of the lightweight feature encoder, F T This is the final output of the lightweight temporal encoder. Concat(·) is the feature concatenation operation, and W is the trainable parameter matrix.

[0044] The multidimensional fusion features F are fused before entering the fully connected layer. d Flattened into a one-dimensional vector, the fully connected layer performs a linear transformation on the one-dimensional vector; the...

[0045] The fully connected layer integrates the previously extracted spatiotemporal features and maps them onto one or more final output values ​​to obtain the predicted values:

[0046] Where Y represents the spatiotemporal features of the input to the last fully connected layer, Linear out This represents a linear function that undergoes a linear transformation.

[0047] Preferably, during training, mean squared error is used as the loss function to measure the deviation between the predicted value and the true value;

[0048] A grid search method is used to perform extensive hyperparameter search. The loss function is optimized by gradient descent and backpropagation algorithms to obtain the optimal parameter combination and thus obtain the trained DLCformer model.

[0049] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0050] 1. The lightweight convolutional self-attention mechanism (LConvformer) of the present invention integrates convolutional operations with self-attention mechanisms, taking into account both local perception capabilities and context modeling capabilities; wherein, convolutional operations are responsible for extracting key local dynamic patterns in the sequence, and self-attention mechanisms are used to capture global dependencies.

[0051] 2. This invention significantly reduces the computational complexity of attention by introducing lightweight design techniques such as tensor broadcasting, reducing it from the traditional quadratic complexity to a near-linear level, thereby improving the model's running efficiency under high-dimensional input.

[0052] 3. This invention employs a parallel encoding structure for both temporal and spatial channels. In the temporal channel, LConvformer (a lightweight ConvFormer mechanism) is used to model the temporal degradation characteristics of each sensor. Similarly, in the spatial channel, LConvformer is introduced to capture the spatial dependency structure between sensor channels at the same time step. The feature representations of the two channels are fused in subsequent stages, thereby achieving collaborative expression and unified modeling of spatiotemporal multidimensional features. This invention uses a dual-channel parallel encoding structure that separates the temporal and sensor dimensions, achieving collaborative modeling of temporal information and spatial coupling relationships. Experimental results show that in device prediction tasks, this invention exhibits significant advantages in both prediction accuracy and computational efficiency, demonstrating good engineering applicability. Attached Figure Description

[0053] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is the overall flowchart of the present invention.

[0055] Figure 2 This is a flowchart illustrating the calculation process of the lightweight timing encoder of the present invention.

[0056] Figure 3 The following are the remaining service life prediction maps of the present invention on the C-MAPSS dataset, where (a) is the remaining service life prediction result map of FD001 in test engine unit 24, (b) is the remaining service life prediction result map of FD002 in test engine unit 80, (c) is the remaining service life prediction result map of FD003 in test engine unit 99, and (d) is the remaining service life prediction result map of FD001 in test engine unit 8. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] like Figure 1 As shown, an embodiment of the present invention provides a device lifetime prediction method based on a dual-channel lightweight convolutional Transformer model. The present invention includes the following steps:

[0059] Step 1: Obtain operational status monitoring data for key equipment.

[0060] In this embodiment, the operational status monitoring data of key equipment is first acquired through a data acquisition system. The key equipment may include, but is not limited to, rotating machinery such as aero-engines, wind turbines, and bearings. The monitoring data is multi-dimensional time-series data, typically derived from multiple sensors installed on key components of the equipment. Sensor types include vibration sensors, temperature sensors, pressure sensors, and speed sensors. For example, in the scenario of aero-engine life prediction, the acquisition system can acquire real-time measurement values ​​from 21 different sensor channels in the engine controller (such as a FADEC system). These measurement values ​​collectively constitute a multi-dimensional time-series data stream reflecting the engine's health status. This step provides the initial data foundation for subsequent equipment condition assessment and life prediction.

[0061] Step 2: Preprocess the running status monitoring data obtained in Step 1 and construct a dataset, dividing the dataset into a training set and a test set.

[0062] Because the original monitoring data has different dimensions, directly inputting it into the model is unlikely to achieve ideal results. Therefore, this step performs a series of preprocessing operations on the data obtained in step S100, which includes the following sub-steps:

[0063] To eliminate dimensional differences between measurements from different sensors and accelerate model training convergence, the data from each sensor channel is normalized. In this embodiment, the Min-MaxNormalization method is used to linearly scale the data from each sensor channel to the interval [0,1] or [-1,1].

[0064] The sample sets generated after processing all devices through the above steps are divided into training and test sets according to a certain proportion or principle. In this embodiment, a portion of the complete lifecycle data of the devices can be used as the training set for learning model parameters; the other portion can be used as the test set to evaluate the final performance of the model.

[0065] Step 3: Construct the DLCformer model: It includes a positional encoding layer, a dual-channel lightweight ConvFormer mechanism, a feature fusion layer, and a fully connected layer connected in sequence; the dual-channel lightweight ConvFormer mechanism includes a lightweight temporal encoder and a lightweight feature encoder connected in parallel.

[0066] This invention sets up an embedding layer in front of the location encoding layer. The embedding layer performs feature dimension transformation and spatial mapping on the preprocessed input sequence in the dataset, making it suitable for subsequent data processing. The location encoding layer embeds spatiotemporal location information into the time series after feature dimension transformation and spatial mapping.

[0067] This invention decouples feature modeling and temporal modeling into independent channels, enabling more effective learning of the complex correlations between spatiotemporal features in the data, thereby significantly improving RUL prediction performance. The outputs of the two channels are fused to form a comprehensive representation, which is then used to achieve RUL prediction through a fully connected layer.

[0068] In modeling sequence data, positional information plays a crucial role in capturing temporal relationships. Since the Transformer architecture itself lacks sequence order awareness, positional encoding techniques are required to inject sequence order information into the model. The positional encoding layer works by constructing a positional encoding matrix based on the frequency domain characteristics of sine and cosine functions. This matrix is ​​then element-wise added to the input time series, effectively embedding the sequence positional information. The specific calculation formula is as follows:

[0069]

[0070] X E =X′+X PE

[0071] In the formula, i1 is the index of the dimension pair, and D... model For model feature dimensions, 2i1 / D model It is the frequency of the i1th dimension pair. X represents the input data after the embedding layer processing, where N is the input feature dimension, T is the total length of the sequence, and pos is the corresponding sequence position index. Even-numbered dimensions (2i1) use sine coding, and odd-numbered dimensions (2i1+1) use cosine coding. Effective embedding of sequence position information is obtained by combining orthogonal functions of different frequencies. E The input matrix after the position coding layer, and the input data X′ after the embedding layer, are compared with the position coding matrix X. PE Direct addition allows the model to know both its own numerical information and its specific position in the time series when calculating the future value of each variable.

[0072] A dual-channel lightweight ConvFormer mechanism is constructed, comprising: a lightweight temporal encoder, focusing on modeling temporal dependencies within the data; and a lightweight feature encoder, handling feature correlations between different sensors at each time step. The feature encoding channel is used to capture spatial correlations between sensors, while the temporal encoding channel models the dynamic evolution of the degradation process.

[0073] like Figure 2 As shown, a lightweight temporal encoder enhances local feature extraction capabilities through convolutional operations. The query matrix Q of a traditional self-attention mechanism... t Key matrix K t Value matrix V t Generating features through global linear transformations is insufficient to effectively capture the local feature patterns required for device degradation detection. To overcome this deficiency, lightweight temporal encoders and lightweight feature encoders employ convolution operations instead of standard linear projection operations, leveraging the local receptive field characteristics of convolution to enhance the interaction capabilities of local features. The specific formula is as follows:

[0074]

[0075] In the formula, This is the input matrix after processing by the position coding layer. Let K be the time score vector, K be the key matrix, V be the value matrix, SoftMax() be the normalization exponential function, ReLU(·) be the activation function, and * be the convolution operation. All are convolution weight matrices with a kernel size of 1, which are automatically learned by the optimization algorithm during the model training process. Specifically, the convolution weight matrix W... τ This is used to fit the signal feature distribution and output the importance of each time step. After normalization by the SoftMax function, a time score vector is obtained.

[0076] Another key drawback of standard self-attention mechanisms lies in their quadratic computational complexity. Traditional self-attention mechanisms involve a complexity of O(N^2). 2Matrix multiplication operations are difficult to apply to the long-term scenarios common in RUL prediction. Therefore, this invention proposes a broadcast strategy to reduce computational overhead. Its core lies in achieving implicit dimensionality expansion through a tensor broadcast mechanism, avoiding explicit data duplication. Specifically, the time score vector... AND key matrix After performing element-wise broadcast multiplication, summation is performed along the time axis to obtain the local feature weight γ, as shown in the following formula:

[0077]

[0078] In the formula, T is the total length of the sequence, and i is the index of the sequence time step. For broadcast operations, this process achieves information propagation across feature dimensions through implicit dimensional expansion. τ i K is the i-th element in the time-scoring vector τ. i It is the i-th row in the key matrix K. After the feature weights and value matrix are fused by a broadcast operation, they are linearly transformed through a convolutional layer with a kernel size of 1 to obtain the final output representation Z, as shown in the following formula:

[0079]

[0080] In the formula, V is the convolution weight matrix. j The feature information related to the j-th node in the value matrix V constitutes a sub-part of the value matrix V, representing a specific location in space. γ j The weight value corresponding to the j-th node in the local feature weight γ is represented by an element of the weight vector γ, which represents the weight assigned to that spatial location. Figure 2 In the context of BroadMul, it represents the proposed broadcast strategy.

[0081] Tensor broadcasting reduces computational complexity to O(N) through element-wise multiplication and broadcasting operations, achieving a significant optimization compared to traditional self-attention mechanisms.

[0082] Lightweight feature encoders are suitable for modeling the correlation between different sensors at each time step, generating feature vectors through convolution operations. The key matrix K′ and value matrix V′ are used to enhance the model's ability to capture local features.

[0083] The input to the lightweight feature encoder consists of the input data X' processed by the embedding layer and its corresponding position encoding matrix X. PE The input matrix X obtained by addition E To accommodate the computational requirements of subsequent convolutional layers, the input dimension needs to be adjusted through a permutation operation, requiring adjustments to the input matrix X. EThe dimensions are replaced. X E T This represents a feature matrix that has incorporated both feature information and temporal location information. `Permute()` is a function that rearranges the dimensions of the tensor, ensuring correct computation when the data is fed into the next layer. The specific formula is as follows:

[0084]

[0085] The specific formula for convolution operation is as follows:

[0086]

[0087] In the formula, For a trainable parameter matrix, The generated feature vectors are K′, the key matrix, and V′, the value matrix.

[0088] Calculate the feature weights γ′ and the output X′ of the broadcast self-attention module. T This calculation process dynamically modulates the key matrix K′ and value matrix V′ using feature weights, and combines this with convolution operations to obtain local feature fusion across sensor features. The specific formula is as follows:

[0089]

[0090] In the formula, For broadcast operation, W T X K is the convolution weight matrix, which is an inherent learnable parameter of a one-dimensional convolutional layer in the model. The final effective value is obtained through iterative optimization learning on the training data. j τ' represents the eigenvector of the new key matrix K' at the j-th node; j ' represents the importance weight of the generated feature vector at the j-th node; γ i ' represents the weight value of feature weight γ' at the i-th position; V i ' represents the eigenvector of the new value matrix V' at the i-th position.

[0091] Each encoder layer contains two core sub-modules: a multi-head lightweight convolutional temporal attention module and a feed-forward network module. To ensure the stability and effectiveness of model training, the output of each sub-module is connected to a residual connection and a layer normalization module (Add & Norm). The layer normalization module is a key component in deep neural networks, its core function being to solve the gradient vanishing problem during deep network training and accelerate model convergence. This module contains two sequentially connected operations: residual connection (Add) and layer normalization (Norm). By combining residual connection and layer normalization, the "Add & Normalization" module in this invention can effectively process the input and output data of the sub-modules, ensuring the stability and efficiency of the deep DLCformer model training process, and ultimately improving the prediction accuracy of the device's remaining lifespan.

[0092] The lightweight temporal encoder comprises a series of lightweight temporal attention layers, a first normalization layer, a series of feedforward networks, and a second normalization layer connected in sequence. The input matrix processed by the position encoding layer is connected to the input residual of the first normalization layer, and the output matrix of the first normalization layer is connected to the input of the second normalization layer.

[0093] The lightweight feature encoder consists of a series of lightweight feature attention layers, a first normalization layer, a series of feedforward networks, and a second normalization layer connected in sequence. The input matrix processed by the position encoding layer is connected to the input residual of the first normalization layer, and the output matrix of the first normalization layer is connected to the input of the second normalization layer.

[0094] In a lightweight temporal encoder, the feedforward network processes features that have undergone lightweight temporal attention aggregation, providing a deeper, non-linear model of the temporal dimension. In a lightweight feature encoder, the feedforward network processes features that have undergone lightweight feature attention aggregation, providing a deeper, non-linear model of the spatial dimension. The computational process of the feedforward network is as follows: it receives lightweight attention and features after the first addition and normalization process, performs non-linear enhancement on them, and finally, the output undergoes a second addition and normalization process to form the final output of either the lightweight temporal encoder or the lightweight feature encoder.

[0095] A feature fusion layer is used to achieve cross-channel information interaction and feature enhancement. To fully integrate the information captured by the dual encoders, a fusion method combining concatenation and linear transformation is used to aggregate the lightweight feature encoder and the lightweight temporal encoder, resulting in feature fusion data. The specific formula is as follows:

[0096] F d =Concat(F S ,F T W

[0097] In the formula, F S The final output of the lightweight feature encoder is derived from the input matrix X. E The result is obtained after stacking multiple lightweight feature attention layers and feedforward networks. T The final output of the lightweight timing encoder is derived from the input data X. E This is obtained after processing through a stack of multiple lightweight temporal attention layers and feedforward networks within the encoder. S and F T It consists of the same, embedded, position-encoded input matrix X. E The final high-level feature representation is obtained after passing through two parallel deep encoders, each dedicated to processing spatial and temporal information. Concat(·) is the feature concatenation operation, and W is a trainable parameter matrix, which serves as the parameters of a linear layer of the model and is automatically learned through the model training process using gradient descent and backpropagation algorithms.

[0098] F fusion feature F d The final RUL prediction value is obtained through a fully connected layer. The fused feature F d Before entering the fully connected layer, the multi-dimensional fused features are flattened into a one-dimensional vector. The fully connected layer then performs a linear transformation on this one-dimensional vector, mapping the high-dimensional feature information to a low-dimensional space. The fully connected layer integrates and maps the highly abstract and complex spatiotemporal features extracted from all the preceding modules onto one or more final output values. The specific formula is as follows:

[0099]

[0100] Where Y represents the high-dimensional feature vector before it is fed into the last fully connected layer. Represents the model's predicted values. Linear out This represents a linear function that takes a high-dimensional feature vector Y, performs a linear transformation to reduce its dimensionality, and maps it to a single scalar output to obtain the predicted value.

[0101] Step 4: Input the training data from the training set into the DLCformer model for training to obtain the trained DLCformer model; input the test data from the test set into the trained DLCformer model to obtain the predicted device lifespan.

[0102] During training, the mean squared error (MSE) is used as the loss function to measure the deviation between the predicted and true values, and a score function is used to characterize the industrial applicability of the model. For both, the smaller the MSE value, the better the model's performance. The specific formula for MSE is as follows. A grid search method is used for extensive hyperparameter search, and the loss function is optimized using the backpropagation algorithm to ensure that the prediction results converge to the optimal solution. The optimal parameter combination of the model is obtained, as shown in Table 1.

[0103]

[0104] In the formula, M is the number of sample points. For predicted values The i2th element, Y i It is the i2th element of the true value.

[0105] Table 1 Optimal parameter combinations for running the DLCformer model

[0106]

[0107] Hidden units are the foundation for a model to represent and understand input data, and their number directly determines the complexity of the model. Choosing 14 is an optimal balance between model performance, computational cost, and the risk of overfitting.

[0108] Stacking multiple layers of feedforward networks enables the model to capture spatial dependencies of different levels and complexities in the data. The feedforward network processes the features after the attention layer and the first addition and normalization fusion.

[0109] The feedforward network in a lightweight temporal encoder operates on features that have undergone temporal attention aggregation. Its task is to non-linearly combine and enhance these temporally dynamic features, setting multiple parameters to capture long-term and complex temporal dependencies and improve prediction accuracy.

[0110] Example

[0111] To verify the effectiveness of the present invention, the following experiment was conducted.

[0112] Experimental environment setup for this invention: running on Windows 10 (64-bit); Intel Core i7-11700 CPU; 16GB RAM; Python 3.8.8; PyTorch-CPU 1.8.1.

[0113] This invention uses the C-MAPSS dataset for algorithm performance analysis. This dataset is open-source turbofan engine data from NASA and is one of the most common datasets in the field of equipment remaining life prediction. The dataset consists of four subsets: FD001, FD002, FD003, and FD004. Each subset contains 26 features, 21 of which are sequence data obtained from sensor monitoring.

[0114] To demonstrate that the algorithm of this invention outperforms other algorithms, the algorithm of this invention and the traditional deep learning method BiLSTM are compared with DCNN, Transformer model, and dual-scale Transformer architecture (DAST). The comparison data is shown in Table 2.

[0115] Table 1 shows the comparison results with other methods on the C-MAPSS dataset.

[0116]

[0117] Analysis shows that BiLSTM achieves RMSE values ​​of 13.65, 23.18, 13.74, and 24.86 on FD001-FD004, respectively. This suboptimal performance can be attributed to its inherent limitations in capturing multi-sensor correlations. The dual-channel algorithm DAST achieves better prediction performance through parallel feature processing. DAST employs two-dimensional self-attention to extract inter-sensor correlations and temporal dependencies, achieving an RMSE of 11.43 on FD001.

[0118] The proposed DLCformer demonstrates significant improvements on most datasets, with the exception of the Score metric for FD003. Specifically, the invention achieves an RMSE of 6.06 and a Score of 148.09 on FD001. This optimal prediction result can be attributed to the introduction of enhanced local feature extraction capabilities, while the dual-channel encoder can process inter-sensor correlations and temporal dependencies in parallel, validating its superior performance in modeling complex spatial correlations and temporal dependencies.

[0119] Figure 3 Analysis of the prediction performance of this invention on four test units of the C-MAPSS dataset shows that the proposed equipment life prediction method exhibits minimal deviation from the actual values ​​across all operating cycles, demonstrating a significant advantage, particularly in capturing key degradation patterns at the end of equipment lifespan. The model maintains stable prediction performance across test units under different operating conditions, validating its strong generalization ability.

[0120] To evaluate the effectiveness of the components in the architecture, this invention conducted ablation experiments on the FD004 subset of the C-MAPSS dataset, which includes multiple operating conditions and composite failure modes. As shown in Table 3, five architecture variants were designed and quantitatively analyzed based on three key metrics: Baseline Encoder, ConvEncoder, Lightweight ConvEncoder (LConvEncoder), Dual-Encoder, and Parallel Lightweight Encoder (DLCformer).

[0121] Table 2 shows the ablation experiments on the FD004 dataset.

[0122]

[0123] Experimental results show that the benchmark encoder achieves an RMSE of 19.93 and a score of 1598.45, confirming the inherent limitations of the standard encoder in capturing key temporal dependencies in RUL prediction. In contrast, ConvEncoder and LConvEncoder enhance local feature extraction capabilities through convolutional layers, thereby improving prediction performance. The dual-channel encoder uses a traditional self-attention mechanism to independently capture sensor associations and temporal dependencies, significantly improving model prediction accuracy with an RMSE of 17.66 and a score of 1518.71, but its computational load increases significantly to 901.19 seconds. The proposed DLCformer, by fusing a dual-channel architecture with the LCformer module, achieves optimal performance with an RMSE of 11.48 and a score of 415.05, demonstrating that this method can effectively and collaboratively capture local and global dependencies.

[0124] Computational efficiency analysis reveals the trade-off between model complexity and inference time. The baseline encoder takes 527.32 seconds to complete inference, ConvEncoder slightly optimizes this to 517.13 seconds, while LConvEncoder, thanks to its lightweight broadcast operation mechanism, significantly reduces the inference time to 294.87 seconds, a reduction of 44.1%. In contrast, the dual-channel encoder incurs a peak computational load of 901.19 seconds due to maintaining parallel processing paths. The DLCformer's inference time is 692.75 seconds, which, although 31.4% higher than the baseline model, is 23.1% lower than the dual-channel encoder. Experimental data confirm that the DLCformer of this invention achieves the optimal balance between prediction accuracy and computational efficiency.

[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A device lifetime prediction method based on a dual-channel lightweight convolutional Transformer model, characterized in that, The steps are as follows: Step 1: Obtain operational status monitoring data for existing key equipment; Step 2: Preprocess the operational status monitoring data; and construct a dataset, dividing the dataset into a training set and a test set; Step 3: Construct the DLCformer model: It includes a positional encoding layer, a dual-channel lightweight ConvFormer mechanism, a feature fusion layer, and a fully connected layer connected in sequence; the dual-channel lightweight ConvFormer mechanism includes a lightweight temporal encoder and a lightweight feature encoder connected in parallel; Step 4: Input the training data from the training set into the DLCformer model for training to obtain the trained DLCformer model; input the test data from the test set into the trained DLCformer model to obtain the predicted device lifespan. The lightweight temporal encoder models the temporal dependencies within the data of the input sequence; Lightweight feature encoders process feature correlations between different sensors within each time step. The lightweight temporal encoder includes multiple lightweight temporal attention layers connected in sequence, and the lightweight feature encoder includes multiple lightweight feature attention layers connected in sequence; the lightweight temporal attention layers and the lightweight feature attention layers are implemented using a lightweight convolutional self-attention mechanism, which includes convolution operations and self-attention mechanisms connected in sequence. The implementation method of the lightweight temporal attention layer of the lightweight temporal encoder is as follows: Replace the standard linear projection operation with convolution operation: ； In the formula, This is the input matrix after processing by the position coding layer. For time-based scoring vectors, The key matrix, Let be the value matrix, and SoftMax() be the normalized exponential function. For activation function, For convolution operations, , , All are convolution weight matrices with a kernel size of 1; convolution weight matrix Used to fit the input matrix The feature distribution is calculated and the importance of each time step is output. The time score vector is then obtained by normalizing the data using the SoftMax function. ; The self-attention mechanism achieves implicit dimensionality expansion through tensor broadcasting, using a time-based scoring vector. AND key matrix After performing element-wise broadcast multiplication, summation is performed along the time axis to obtain the local feature weights. ; Local feature weights AND-value matrix After broadcast fusion, a linear transformation is performed through a convolutional layer with a kernel size of 1 to obtain the final output representation. .

2. The device lifetime prediction method based on a dual-channel lightweight convolutional Transformer model according to claim 1, characterized in that, The location encoding layer is front-end set with an embedding layer, which performs feature dimension transformation and spatial mapping on the preprocessed input sequence; the location encoding layer embeds spatiotemporal location information into the time sequence after feature dimension transformation and spatial mapping.

3. The device lifetime prediction method based on a dual-channel lightweight convolutional Transformer model according to claim 1 or 2, characterized in that, The method for data processing in the location coding layer is as follows: ； in, For the input data of the input position coding layer, For the input feature dimension, The total length of the sequence; This is a position coding matrix based on the frequency domain characteristics of sine and cosine functions; The location encoding matrix is ； In the formula, i1 is the index of the dimension pair. For model feature dimensions, 2i1 / D model It is the frequency of the i1th dimension pair; This is the position index of the corresponding sequence.

4. The device lifetime prediction method based on a dual-channel lightweight convolutional Transformer model according to claim 1, characterized in that, The implementation method of the lightweight feature attention layer of the lightweight feature encoder is as follows: The input matrix is ​​subjected to permutation operations. The input matrix is ​​obtained by permuting the dimensions to obtain a matrix that integrates feature information and temporal location information. ; Feature vectors are generated through convolution operations. Key matrix Value matrix The convolution operation is: ； In the formula, , , For a trainable parameter matrix, For the generated feature vector, The key matrix, It is a value matrix; Feature vectors generated by tensor broadcasting mechanism AND key matrix After performing element-wise broadcast multiplication, the feature weights are obtained by summing along the nodes. ; through feature weights value matrix Dynamic modulation is performed, and convolution operations are combined to obtain local fusion features across sensor features. .

5. The device lifetime prediction method based on a dual-channel lightweight convolutional Transformer model according to claim 4, characterized in that, The local feature weights are: ; In the formula, T is the total length of the sequence, and i is the index of the sequence time step. For broadcast operation, It is a time-based scoring vector The i-th element in It is a key matrix The i-th row in; The final output representation for ; In the formula, The convolution weight matrix is... Value matrix Feature information related to the j-th node Represents local feature weights The weight value corresponding to the j-th node; The input matrix Permute() is a function that rearranges the dimensions of a tensor. The feature weights The local fusion features ; In the formula, The convolution weight matrix is... Key matrix The feature vector at the j-th node; This indicates that the element of the generated feature vector at the j-th node is... Represents feature weights The weight value at the i-th position; Value matrix The feature vector at the i-th position.

6. The device lifetime prediction method based on a dual-channel lightweight convolutional Transformer model according to claim 4 or 5, characterized in that, Both the lightweight temporal encoder and the lightweight feature encoder further include a first normalization layer, a multi-layer feedforward network, and a second normalization layer connected in sequence. The first normalization layer is connected to the output of the last lightweight temporal attention layer of the lightweight temporal encoder or the output of the last lightweight feature attention layer of the lightweight feature encoder. The input matrix processed by the position encoding layer is connected to the residual of the input of the first normalization layer, and the output matrix of the first normalization layer is connected to the input of the second normalization layer. The calculation process of the feedforward network is as follows: it receives the output of the lightweight temporal attention layer or the lightweight feature attention layer and the features after the first addition and normalization process, performs nonlinear enhancement, and then the output is normalized again by the second addition and normalization to form the final output of the entire lightweight temporal encoder or lightweight feature encoder.

7. The device lifetime prediction method based on a dual-channel lightweight convolutional Transformer model according to claim 6, characterized in that, The feature fusion layer uses a combination of splicing operations and linear transformations to aggregate the final outputs from the lightweight feature encoder and the lightweight temporal encoder, resulting in fused features. ; In the formula, This is the final output of the lightweight feature encoder. For the final output of a lightweight timing encoder, For feature splicing operations, It is a trainable parameter matrix; Multidimensional fusion features are incorporated before entering the fully connected layer. The vector is flattened into a one-dimensional vector, and a fully connected layer performs a linear transformation on the one-dimensional vector. This fully connected layer integrates the previously extracted spatiotemporal features and maps them to one or more final output values ​​to obtain the predicted value. ; Where Y represents the spatiotemporal features of the input to the last fully connected layer. This represents a linear function that undergoes a linear transformation.

8. The device lifetime prediction method based on a dual-channel lightweight convolutional Transformer model according to claim 7, characterized in that, During training, mean squared error is used as the loss function to measure the deviation between the predicted value and the true value; A grid search method is used to perform extensive hyperparameter search. The loss function is optimized by gradient descent and backpropagation algorithms to obtain the optimal parameter combination and thus obtain the trained DLCformer model.

Images