Method for predicting residual life of mechanical equipment based on 3Dform

Abstract

The invention provides a mechanical equipment residual life prediction method based on 3Dform, and the method comprises the steps: S1, constructing a 3Dform prediction model which comprises an input data preprocessing module, a 3D convolution and channel attention module and a prediction module based on sequence decomposition, which are connected in sequence; s2, acquiring time series data of the mechanical equipment sensor, and preprocessing the time series data of the mechanical equipment sensor by using an input data preprocessing module to obtain 3D data about the mechanical equipment sensor; s3, performing 3D convolution operation on the 3D data about the mechanical equipment sensor by using a 3D convolution and channel attention module to obtain spatial-temporal features, and enhancing the spatial-temporal features to obtain an enhanced feature map; and S4, modeling seasonal components according to the enhanced feature pattern by using a prediction module based on sequence decomposition, and performing RUL prediction based on the seasonal components. According to the method, the spatial-temporal characteristics of the state change of the mechanical equipment are more accurately captured, so that the accuracy of RUL prediction is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent operation and maintenance of mechanical equipment, and in particular to a method for predicting the remaining life of mechanical equipment. Background Art

[0002] Remaining useful life prediction (RUL) is crucial in predictive maintenance, especially in high-value and safety-critical components. Accurate RUL prediction can schedule preventive maintenance, reduce unplanned downtime, and optimize resource allocation. In the context of Industry 4.0, the increasing abundance of sensor data and advances in artificial intelligence technology have opened up new opportunities for data-based RUL prediction methods. However, the complex and nonlinear degradation patterns of mechanical equipment, as well as noise and uncertainty, bring significant challenges to the application of traditional prediction methods.

[0003] Deep learning based methods have attracted significant attention in recent years for the task of predicting remaining useful life (RUL). These methods leverage the powerful feature learning and representation capabilities of deep neural networks to model complex patterns in the data. Convolutional neural networks (CNNs) are often used to extract spatial features from sensor data, while recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) are used to capture temporal dependencies. In addition, attention mechanisms are integrated to direct attention to the most critical features and time steps.

[0004] In recent years, Transformer-based methods have rapidly emerged in various fields, including natural language processing, computer vision, and time series prediction. Its main advantage lies in the self-attention mechanism, which excels in capturing long-range dependencies in sequence data. This enables the Transformer to effectively model complex patterns by learning the relative importance of different parts of the input sequence, thereby potentially improving the accuracy and robustness of predictions in RUL applications. This is particularly valuable in RUL prediction, as degradation patterns may exhibit long-term dependencies and relevant information may be distributed in different time steps. Several studies have explored the application of Transformer in RUL prediction. Ren et al. introduced a Transformer-based network (DLformer) that adaptively learns dynamic feature representations for efficient RUL prediction, and adopts a dynamic sequence length method and feature reuse mechanism to reduce the amount of computation while maintaining accuracy. Peng et al. developed a multi-scale temporal convolutional Transformer that combines convolutional self-attention and temporal convolution to capture long-term patterns and local information from sensor data. Despite promising results, Transformer-based RUL prediction methods still face challenges. Transformers often face challenges in extracting local spatiotemporal features that are crucial for RUL prediction. Summary of the invention

[0005] In view of the technical problem that the existing technology has insufficient ability to extract local spatiotemporal features that are crucial for RUL prediction, the present invention proposes a remaining life prediction method for mechanical equipment based on 3Dformer. The method synergistically combines the advantages of 3D convolutional neural network (3DCNN) and Transformer, uses 3DCNN to simultaneously extract localized spatiotemporal features from raw sensor data, considers the time dimension and feature dimension, integrates a channel attention mechanism in 3DCNN to adaptively enhance the most informative features and suppress irrelevant features, introduces a sequence decomposition structure in the encoder of Transformer to model trend and periodic components respectively, so as to better capture the multi-scale characteristics of time series data, and adopts a self-attention mechanism to model the long-term dependencies in the decomposed components, so that the model can focus on the most relevant information in RUL prediction.

[0006] In order to achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] A method for predicting the remaining life of mechanical equipment based on 3Dformer, comprising the steps of:

[0008] S1: Construct a 3Dformer prediction model, including a sequentially connected input data preprocessing module, a 3D convolution and channel attention module, and a prediction module based on sequence decomposition;

[0009] S2: Acquire time series data of mechanical equipment sensors, and use an input data preprocessing module to preprocess the time series data of mechanical equipment sensors to obtain 3D data about the mechanical equipment sensors;

[0010] S3: Use 3D convolution and channel attention modules to perform 3D convolution operations on the 3D data of the mechanical equipment sensor to obtain spatiotemporal features, and enhance the spatiotemporal features to obtain enhanced feature maps;

[0011] S4: The seasonal component is modeled according to the enhanced feature map using the prediction module based on sequence decomposition, and RUL prediction is performed based on the seasonal component.

[0012] The 3D convolution and channel attention module includes a multi-layer 3D convolution module and a channel attention module connected in sequence, and a three-dimensional adaptive average pooling layer is arranged between every two 3D convolution layers in the multi-layer 3D convolution module; the multi-layer 3D convolution module is connected to the input data preprocessing module, and the channel attention module is connected to the prediction module based on sequence decomposition.

[0013] The prediction module based on sequence decomposition includes a sequence decomposition layer I, the input of the sequence decomposition layer I is connected to the channel attention module, the sequence decomposition layer I is respectively connected to the multi-head attention mechanism layer and the Add&Norm layer II, the sequence decomposition layer I and the multi-head attention mechanism layer are both connected to the Add&Norm layer I, the Add&Norm layer I is connected to the sequence decomposition layer II, the sequence decomposition layer II is respectively connected to the Add&Norm layer II, the feedforward network layer I and the Add&Norm layer III, the Add&Norm layer III is connected to the Add&Norm layer IV, the Add&Norm layer II is connected to the Add&Norm layer IV through the feedforward network layer II, and the Add&Norm layer IV is connected to the fully connected layer.

[0014] The method for obtaining 3D data about mechanical equipment sensors described in step S2 is: normalize the time series data of the mechanical equipment sensors using minimum-maximum scaling; use sliding window technology to divide the normalized time series data of the mechanical equipment sensors into subsequences and form multivariate time series sample data of the sensors; and use time lag shift technology and 1D convolution to reconstruct the multivariate time series sample data of the sensors in turn to generate a 3D tensor for representing the 3D data about the mechanical equipment sensors.

[0015] The method for obtaining the spatiotemporal features described in step S3 is: inputting the 3D tensor used to represent the 3D data of the mechanical equipment sensor into the multi-layer 3D convolution module, and obtaining the 3D tensor X used to represent the spatiotemporal features of the mechanical equipment sensor through layer-by-layer 3D convolution operations and 3D pooling operations. (L) ∈R C×S×H×W .

[0016] The calculation method of the 3D convolution operation is:

[0017]

[0018] in, represents the input 3D tensor of the k-1th layer of the 3D convolution module, C (k-1) is the number of output channels of the k-1th layer, W (k-1) represents the width of the 3D tensor, H (k-1) Represents the height of the 3D tensor, S (k-1) represents the time dimension of the 3D tensor; * represents the 3D convolution operation, σ represents the sigmoid activation function, and These are the learnable parameters of the kth layer of the 3D convolution module.

[0019] The method for enhancing the spatiotemporal features to obtain an enhanced feature map is: 3D tensor X (L)∈R C×S×H×W After global average pooling and maximum pooling, the full average pooling features and the maximum pooling features are obtained. Then, the full average pooling features and the maximum pooling features are processed by a multi-layer perceptron to obtain the channel attention matrix M C ∈R C×1×1×1 :

[0020] M C (X (L) )=σ(MLP(Avgpool(X (L) ))+MLP(Maxpool(X (L) )))

[0021] in, and Respectively represent the full average pooling feature and the maximum pooling feature; W 0 and W 1 represents the trainable parameter, σ represents the sigmoid activation function, MLP() represents the multi-layer perceptron operation function, Avgpool() represents the global average pooling operation function, and Maxpool() represents the maximum pooling operation function; then, the obtained channel attention matrix M C Applied to the input feature map X (L) , generate enhanced feature map F′∈R C × S ×H× W :

[0022] F′=M c (X (L) )⊙X (L) ;

[0023] Among them, ⊙ represents element-wise multiplication.

[0024] Step S4 is as follows: the prediction module based on sequence decomposition takes the enhanced feature map F′ as input, and uses the sequence decomposition layer I to obtain the seasonal component F from the enhanced feature map F′. s ' 1 ; Through the multi-head attention mechanism layer from the seasonal component F s ' 1 The 3D tensor MultiHead(Q t ,K t ,V t );The seasonal component F is added through Add&Norm layer I s ' 1 and 3D tensor MultiHead(Q t ,K t ,V t) is subjected to residual connection and normalization processing to obtain normalized information I; the seasonal component F is extracted from the normalized information I through the sequence decomposition layer II s ' 2 Then, the seasonal component F s ' 1 and seasonal component F s ' 2 The fusion and normalization processing are performed to obtain the normalized information II, and the feature conversion is performed through the feedforward network II to obtain the feature conversion result I; the seasonal component F is converted through the feedforward network I s ' 2 Perform feature transformation to obtain feature transformation result II; use Add&Norm layer III to transform the seasonal component F s ' 2 The feature conversion result Ⅱ is residually connected and normalized to obtain normalized information Ⅲ; the feature conversion result Ⅰ and the normalized information Ⅲ are fused and normalized using the Add&Norm layer Ⅳ and then sent to the fully connected layer for integrated regression to output the RUL prediction value.

[0025] The method for obtaining seasonal components is as follows: using multiple filters of different sizes to extract multiple trend items from input data; aggregating the extracted multiple trend items into a final trend component through weights based on the correlation between the multiple trend items and the input data; and obtaining seasonal components based on the difference between the trend component and the input data.

[0026] The method for extracting multiple trend items from input data is as follows:

[0027]

[0028] Among them, d model Indicates the number of average pooling filters, Padding d () indicates the filling operation function, Avgpol d () represents the average pooling operation function, Concat represents the concatenation operation, and F″(F) represents the extracted multiple trend items;

[0029] The method of aggregating the final trend components through weights is:

[0030] F t ′=Softmax(L(F))F″(F)

[0031] Among them, L() represents the fully connected layer operation, Softmax() is the activation function, Softmax(L(F)) represents the weight of the trend item used for fusion extraction, and F t '' is the trend component;

[0032] The calculation method of the seasonal component is: s ′=FF t ′.

[0033] Beneficial effects of the present invention:

[0034] The introduction of 3D convolution can capture spatiotemporal features from multi-sensor data. This method not only considers changes in the time dimension, but also considers the spatial correlation between different sensors, providing a more comprehensive data representation; the channel attention mechanism is integrated after 3DCNN to automatically identify and strengthen the feature channels that are most critical for RUL prediction, reducing unnecessary computational burden and improving the learning efficiency of the model, adaptively enhancing the most informative features and suppressing irrelevant features; a decomposition-integrated encoder is introduced in the Transformer network to model the trend and periodicity of the time series and enhance the capture of multi-scale characteristics; the present invention more accurately captures the spatiotemporal characteristics of the state changes of mechanical equipment, thereby improving the accuracy of RUL prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 This is a model diagram of the RUL prediction method based on 3DFormer of the present invention.

[0037] Figure 2 Schematic diagram of the multi-head attention mechanism of the present invention.

[0038] Figure 3 Schematic diagram of the 3D convolution operation of the present invention.

[0039] Figure 4 Schematic diagram of the channel attention mechanism (CA) of the present invention.

[0040] Figure 5 It is a schematic diagram of the RUL prediction results of four test engine units of the present invention, Figure (a) is the data set FD001 corresponding to the engine unit; Figure (b) is the data set FD002 corresponding to the engine unit, Figure (c) is the data set FD003 corresponding to the engine unit, and Figure (d) is the data set FD004 corresponding to the engine unit. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Embodiment 1:

[0043] A 3Dformer-based method for predicting the remaining life of mechanical equipment, such as Figure 1 As shown, the steps include:

[0044] S1: Build a 3Dformer prediction model.

[0045] The prediction model includes an input data preprocessing module, a 3D convolution and channel attention module, and a prediction module based on sequence decomposition, which are connected in sequence;

[0046] The input data preprocessing module is used to obtain the time series data of the mechanical equipment sensor. The input data preprocessing module is used to preprocess the time series data of the mechanical equipment sensor to obtain 3D data about the mechanical equipment sensor.

[0047] The 3D convolution and channel attention modules are used to perform 3D convolution operations on the 3D data of the mechanical equipment sensor to obtain spatiotemporal features, and enhance the spatiotemporal features to obtain enhanced feature maps.

[0048] The prediction module based on sequence decomposition is used to model the seasonal component according to the enhanced feature map and perform RUL prediction based on the seasonal component.

[0049] The 3D convolution and channel attention module includes a multi-layer 3D convolution module and a channel attention module connected in sequence; a three-dimensional adaptive average pooling layer is provided between every two 3D convolution layers in the multi-layer 3D convolution module; the multi-layer 3D convolution module is connected to the input data preprocessing module, and the multi-layer 3D convolution module is used to perform a 3D convolution operation on the 3D data of the mechanical equipment sensor to obtain spatiotemporal features. The channel attention module is connected to the prediction module based on sequence decomposition, and the channel attention module is used to enhance the spatiotemporal features to obtain an enhanced feature map.

[0050] The prediction module based on sequence decomposition includes a sequence decomposition layer I, the input of which is connected to the channel attention module, the sequence decomposition layer I is connected to the multi-head attention mechanism layer and the Add&Norm layer II respectively, the sequence decomposition layer I and the multi-head attention mechanism layer are both connected to the Add&Norm layer I, the Add&Norm layer I is connected to the sequence decomposition layer II, the sequence decomposition layer II is connected to the Add&Norm layer II, the feedforward network layer I and the Add&Norm layer III respectively, the feedforward network layer I is connected to the Add&Norm layer III, the Add&Norm layer III is connected to the Add&Norm layer IV, the Add&Norm layer II is connected to the Add&Norm layer IV through the feedforward network layer II, and the Add&Norm layer IV is connected to the fully connected layer.

[0051] S2: Acquire the time series data of the mechanical equipment sensor, and use the input data preprocessing module to preprocess the time series data of the mechanical equipment sensor to obtain 3D data about the mechanical equipment sensor.

[0052] The mechanical equipment sensor time series data is normalized using min-max scaling. The measurement value of each sensor is scaled to the range of [0,1] using the following formula:

[0053]

[0054] in, is the normalized data, max(X i ) and min(X i ) represent the maximum and minimum values ​​of the measurements in each sensor, respectively.

[0055] In order to prepare data for the 3DFormer model, a sliding window technique is used. The measured values ​​in the sensor are divided into subsequences of fixed length through a sliding window of size w. Each subsequence represents a sequence of sensor measurements within w time steps, which are used as input samples of the 3DFormer model. The RUL value corresponding to each subsequence is determined by the number of remaining cycles from the last time step of the window to the fault.

[0056] The time series data of mechanical equipment sensors is represented by S = {x t ,x t+1 ,...,x t+N-1}, N is the time step of the time series data, where x t is the time series data of m sensors recorded at time t, x t ={x 1,(t) ,x 2,(t) ,...,x m,(t)}, T iis the sequence length of the time series data of the mechanical equipment sensor. The sliding window technology is used for the time series data S, and the sliding window w is T i When the subsequence of the mechanical equipment sensor time series data is That is, the multivariate time series sample data of the sensor, x t ={x 1,(t) ,x 2,(t) ,...,x m,(t)}.

[0057] Finally, the multivariate time series sample data X of the sensor is reconstructed by using the time lag shift technique and 1D convolution to generate a 3D tensor, as shown in Figure 1 As shown in the input data preprocessing module.

[0058] S3: Use 3D convolution and channel attention modules to perform 3D convolution operations on the 3D data of mechanical equipment sensors to obtain spatiotemporal features, and enhance the spatiotemporal features to obtain enhanced feature maps.

[0059] Unlike traditional one-dimensional (1D) or two-dimensional (2D) convolutions, 3DCNN considers both the time dimension and the feature dimension, enabling it to extract more comprehensive spatiotemporal features. By organizing the input samples into a 3D tensor, two key correlations emerge: the spatial cross-correlation between different variables and the temporal autocorrelation within each variable. 3DCNN addresses this challenge by applying convolution operations on both the time and feature dimensions, enabling it to capture meaningful spatiotemporal patterns. This approach enables the model to capture the complex interactions between different sensors and their temporal evolution, providing more robust and informative feature representations for subsequent tasks.

[0060] The method for performing 3D convolution operation is: Figure 3 As shown, the 3D tensor is input into the multi-layer 3D convolution module, and the spatiotemporal features of the mechanical equipment sensor are obtained through layer-by-layer 3D convolution operations and 3D pooling operations. The calculation formula is:

[0061]

[0062] in, represents the input 3D tensor of the 3D convolution module of the kth layer, C (k-1) is the number of output channels of the k-1th layer 3D convolution module, W (k-1) represents the width of the 3D tensor, H (k-1) Represents the height of the 3D tensor, S (k-1) represents the time dimension of the 3D tensor; * represents the 3D convolution operation, σ represents the sigmoid activation function, and is the learnable parameter of the k-th layer 3D convolution module. After L layers of 3D convolution, a 3D tensor X can be obtained to represent the spatiotemporal characteristics of the mechanical equipment sensor. (L) ∈R C×S×H×W .

[0063] After completing the 3D convolution operation, a channel attention (CA) mechanism is applied to adaptively weight the importance of different sensor channels. This mechanism helps the model focus on the most informative features, thereby improving the accuracy of RUL prediction.

[0064] The method for enhancing the spatiotemporal features to obtain the enhanced feature map is:

[0065] like Figure 4 As shown, the 3D tensor X (L) ∈R C×S×H×W After global average pooling and maximum pooling, two sets of channel feature information are obtained: maximum pooling feature And full average pooling features Then, the multi-layer perceptron is used to process the full average pooling features and the maximum pooling features to obtain the channel attention matrix M C ∈R C×1×1×1 The calculation formula of the channel attention matrix is ​​as follows:

[0066]

[0067] in, and Represent the global average pooling feature and the maximum pooling feature respectively. 0 and W 1 represents the trainable parameter, σ represents the sigmoid activation function, MLP() represents the multi-layer perceptron operation function, Avgpool() represents the global average pooling operation function, and Maxpool() represents the maximum pooling operation function. Then, the obtained channel attention matrix M C Applied to the input feature map X (L) , generate enhanced feature map F′∈R C × S ×H× W :

[0068] F′=M c (X (L) )⊙X (L) );

[0069] Among them, ⊙ represents element-wise multiplication.

[0070] S4: The seasonal component is modeled according to the enhanced feature map using the prediction module based on sequence decomposition, and RUL prediction is performed based on the seasonal component.

[0071] like Figure 1 As shown in the prediction module, the specific process of step S4 is as follows: the prediction module based on sequence decomposition takes the enhanced feature map F′ as input, and uses the sequence decomposition layer I to obtain the seasonal component F from the enhanced feature map F′. s ' 1 ; Through the multi-head attention mechanism layer from the seasonal component F s ' 1 The 3D tensor MultiHead(Q t ,K t ,V t );The seasonal component F is added through Add&Norm layer I s ' 1 and 3D tensor MultiHead(Q t ,K t ,V t ) is used for residual connection and normalization processing to ensure that the output of each layer contains both the learning content of the current layer and the input information, and the normalized information I is obtained; the seasonal component F is extracted from the normalized information I through the sequence decomposition layer II s ' 2 Then, the seasonal component F s ' 1 and seasonal component F s ' 2 The fusion and normalization processing are performed to obtain the normalized information II, and the feature conversion is performed through the feedforward network II to obtain the feature conversion result I; the seasonal component F is converted through the feedforward network I s ' 2 Perform feature conversion to obtain feature conversion result II; use Add&Norm layer III to convert seasonal component seasonal component F s ' 2 The feature conversion result Ⅱ is residually connected and normalized to obtain normalized information Ⅲ; the feature conversion result Ⅰ and the normalized information Ⅲ are fused and normalized using the Add&Norm layer Ⅳ and then sent to the fully connected layer for integrated regression to output the RUL prediction value.

[0072] In order to effectively capture the multi-scale characteristics of time series data, the 3DFormer model introduces a sequence decomposition method in the encoder of the Transformer network. This enables the model to capture and model trend components and seasonal components separately. Generally, average pooling using a fixed window is a common method for extracting trend terms. However, this method encounters difficulties when dealing with complex periodic patterns in real data. Generally, shorter window lengths are more sensitive to mutations and significant features in time series. In order to solve this problem, the processing method of the 3Dformer model of the present invention is as follows:

[0073] The method for obtaining seasonal components is: using a set of multiple filters of different sizes to extract multiple trend items from the input data, and obtaining:

[0074]

[0075] Among them, d model Indicates the number of average pooling filters, Padding d () represents the padding operation function, which is used to maintain the sequence length. d () represents the average pooling operation function, Concat represents the concatenation operation, and F″(F) represents the extracted multiple trend items.

[0076] Then, based on the correlation between the extracted trend item data and the input data F, the input data F of the sequence decomposition layer I is the enhanced feature map F′, and the input data F of the sequence decomposition layer II is the normalized information I, which are aggregated into the final trend component through weights:

[0077] F t ′=Softmax(L(F))F″(F)

[0078] Among them, L() represents the fully connected layer operation, Softmax() is the activation function, Softmax(L(F)) represents the weight of the trend item used for fusion extraction, and F t '' is the trend component.

[0079] Obtain the seasonal component F based on the difference between the trend component and the input data s ′:

[0080] F s ′=FF t '

[0081] like Figure 2 As shown, the calculation process of the multi-head attention mechanism layer is: seasonal component F s ′After linear transformation, the corresponding query Q is obtained t , key K t Sum value V t ; The query Q is converted into t , key K t Sum value V t Mapping into multiple low-dimensional spaces allows the attention in each mapping to focus on different sequence segments, thereby obtaining a 3D tensor that is used to represent the rich features of each position in the seasonal component.

[0082] The formula for multi-head self-attention is as follows:

[0083] head i=Attention(Q t ,K t ,V t ) i

[0084]

[0085] in, represents the output weight matrix, h is the number of attention heads, head i is the output of each attention head i, D model is the feature dimension.

[0086] The structure of the feedforward neural network (FFN) consists of two linear transformations, with a rectified linear unit (ReLU activation function) activation function inserted in the middle. The calculation formula of the feedforward neural network (FFN) is:

[0087] FFN(x)=ReLU(xW 1 +b 1 )W 2 +b 2

[0088] Among them, W 1 and W 2 is the weight matrix, b 1 and b 2 is the bias, x is the input of the feedforward neural network, that is, F s ' 2 Seasonal component or normalization information II. Feedforward neural networks introduce nonlinearity and enhance the model's ability to learn complex relationships between attention features.

[0089] The fully connected layer processes the output to generate the RUL prediction, as follows:

[0090] y i =σ(W i X′+b i )

[0091] Wherein, σ represents an activation function, X′ represents the output of Add&Norm layer IV in the present invention; W i and b i Represent the weight matrix and bias vector respectively, y i Represents the predicted RUL output of the fully connected layer.

[0092] Example 2

[0093] A method for predicting the remaining life of mechanical equipment based on 3Dformer.

[0094] In this embodiment, the mechanical equipment sensor time series data adopts the C-MAPSS data set. The C-MAPSS data set includes four subsets: FD001, FD002, FD003 and FD004, each of which represents the performance of a turbofan engine under different operating conditions and failure modes, and each subset has different characteristics, as shown in Table 1. Each subset contains multivariate time series data from 21 sensors, which are recorded at fixed intervals until an engine failure occurs.

[0095] The FD001 subset originally contained time series data from 21 sensors. However, analysis showed that during the monitoring period, the readings of sensors 1, 5, 6, 10, 16, 18, and 19 remained constant and contributed little information to the RUL prediction. Therefore, this example focuses the prediction model on the remaining 14 sensors that exhibit variability and may provide valuable prediction information.

[0096] Hyperparameters and implementation details: The 3DFormer model is implemented using PyTorch. The hyperparameters are optimized by grid search to achieve the best performance on each C-MAPSS subset. The final architecture parameters are shown in Table 1. Key configuration details include: the sliding time window length is 40 for the FD001 dataset, while the sliding time window length is 20 for the FD002, FD003, and FD004 datasets. We adopt the modified Adam optimizer for training and use RMSE as the loss function. The learning rate is set to 0.001, the number of training rounds is 80, and the batch size is 128. A dropout rate of 0.1 is applied. The training and testing processes are carried out in the following environment: operating system: Windows 10 Education Edition (64-bit); CPU: 11th Generation Intel Core i7-11700@2.50GHz; memory: 16.0GB; Python version is 3.8.8; PyTorch-CPU version is 1.8.1.

[0097] To evaluate the performance of the 3DFormer model, the root mean square error (RMSE) and the scoring function are used as the main indicators. RMSE measures the average difference between the predicted RUL value and the true RUL value, while the scoring function evaluates the performance of the model based on the asymmetric penalty for overestimation and underestimation of RUL. The definitions of the root mean square error (RMSE) and the scoring function are as follows:

[0098]

[0099]

[0100] Where n is the number of input samples, y i ′ is the predicted RUL, y iis the true RUL of the i-th input sample.

[0101] The other structures and principles are the same as those in Example 1.

[0102] Experimental results:

[0103] This embodiment compares the model DAST, the model DSFormer and the 3Dformer proposed in the present invention.

[0104] Table 2 shows that the proposed 3DFormer outperforms the state-of-the-art methods on the C-MAPSS dataset. DSFormer is an enhanced Autoformer model that can effectively capture multi-scale temporal patterns and enhance feature representation. The proposed 3DFormer outperforms the model DSFormer on most datasets, except for the FD003 subset, where it achieves the lowest RMSE and score values. The superior performance of the proposed 3DFormer can be attributed to the combination of 3D CNN for spatiotemporal feature extraction and sequence decomposition for capturing long-term temporal dependencies, thereby achieving more accurate and reliable remaining useful life prediction.

[0105] Table 1

[0106]

[0107]

[0108] Table 2 Comparison results with other methods on the C-MAPSS dataset

[0109]

[0110] Figure 5 A visual analysis of the remaining useful life (RUL) prediction results for four test engine units in the C-MAPSS dataset is shown. The figure compares the predicted RUL values ​​of the proposed method with the true values, demonstrating the model's ability to accurately estimate the remaining useful life. The model of the present invention closely follows the true RUL trajectory in all four cases, demonstrating its effectiveness in capturing degradation patterns. In contrast, baseline methods often have problems with overestimation or underestimation, resulting in unreliable prediction results. These visualizations demonstrate the innovativeness of the proposed method, which synergistically combines multiple modules to extract information-rich features and model long-range dependencies. The accurate and reliable prediction results in different engine units demonstrate the superiority of the proposed method in dealing with the complexity of RUL prediction in real-world scenarios.

Claims

1. A method for predicting the remaining life of mechanical equipment based on 3Dformer, characterized in that: Includes steps: S1: Construct a 3Dformer prediction model, including a sequentially connected input data preprocessing module, a 3D convolution and channel attention module, and a prediction module based on sequence decomposition; S2: Acquire time series data of mechanical equipment sensors, and use an input data preprocessing module to preprocess the time series data of mechanical equipment sensors to obtain 3D data about the mechanical equipment sensors; S3: Use 3D convolution and channel attention modules to perform 3D convolution operations on the 3D data of the mechanical equipment sensor to obtain spatiotemporal features, and enhance the spatiotemporal features to obtain enhanced feature maps; S4: The seasonal component is modeled according to the enhanced feature map using the prediction module based on sequence decomposition, and RUL prediction is performed based on the seasonal component.

2. The method for predicting the remaining service life of mechanical equipment based on 3Dformer according to claim 1, characterized in that: The 3D convolution and channel attention module comprises a multi-layer 3D convolution module and a channel attention module connected in sequence, wherein a three-dimensional adaptive average pooling layer is arranged between every two 3D convolution layers of the multi-layer 3D convolution module; the multi-layer 3D convolution module is connected to the input data preprocessing module. The channel attention module is connected to the prediction module based on sequence decomposition.

3. The method for predicting the remaining service life of mechanical equipment based on 3Dformer according to claim 2, characterized in that: The prediction module based on sequence decomposition includes a sequence decomposition layer I, the input of the sequence decomposition layer I is connected to the channel attention module, the sequence decomposition layer I is respectively connected to the multi-head attention mechanism layer and the Add&Norm layer II, the sequence decomposition layer I and the multi-head attention mechanism layer are both connected to the Add&Norm layer I, the Add&Norm layer I is connected to the sequence decomposition layer II, the sequence decomposition layer II is respectively connected to the Add&Norm layer II, the feedforward network layer I and the Add&Norm layer III, the Add&Norm layer III is connected to the Add&Norm layer IV, the Add&Norm layer II is connected to the Add&Norm layer IV through the feedforward network layer II, and the Add&Norm layer IV is connected to the fully connected layer.

4. The method for predicting the remaining service life of mechanical equipment based on 3Dformer according to any one of claims 1 to 3, characterized in that: The method for obtaining 3D data about mechanical equipment sensors described in step S2 is: normalizing the time series data of the mechanical equipment sensors using minimum-maximum scaling; using sliding window technology to divide the normalized time series data of the mechanical equipment sensors into subsequences and form multivariate time series sample data of the sensors; and using 1D convolution and time lag shift technology to reconstruct the multivariate time series sample data of the sensors to generate a 3D tensor for representing the 3D data about the mechanical equipment sensors.

5. The method for predicting the remaining service life of mechanical equipment based on 3Dformer according to claim 4, characterized in that: The method for obtaining the spatiotemporal features described in step S3 is: inputting the 3D tensor used to represent the 3D data of the mechanical equipment sensor into the multi-layer 3D convolution module, and obtaining the 3D tensor X used to represent the spatiotemporal features of the mechanical equipment sensor through layer-by-layer 3D convolution operations and 3D pooling operations. (L) .

6. The method for predicting the remaining service life of mechanical equipment based on 3Dformer according to claim 5, characterized in that: The calculation method of the 3D convolution operation is: in, represents the input 3D tensor of the kth layer of the 3D convolution module, C (k-1) is the number of output channels of the k-1th layer, W (k-1) represents the width of the 3D tensor, H (k-1) Represents the height of the 3D tensor, S (k-1) represents the time dimension of the 3D tensor; * represents the 3D convolution operation, σ represents the sigmoid activation function, and is the learnable parameter of the kth layer of the 3D convolution module.

7. The method for predicting the remaining service life of mechanical equipment based on 3Dformer according to claim 6, characterized in that: The method for enhancing the spatiotemporal features to obtain an enhanced feature map is: 3D tensor X (L) After global average pooling and maximum pooling, the full average pooling features and the maximum pooling features are obtained. Then, the full average pooling features and the maximum pooling features are processed by a multi-layer perceptron to obtain the channel attention matrix M C ∈R C×1×1×1 : M C (X (L) )=σ(MLP(Avgpool(X (L) ))+MLP(Maxpool(X (L) ))) in, and Represent the full average pooling feature and the maximum pooling feature respectively. W0 and W1 represent trainable parameters, σ represents the sigmoid activation function, MLP() represents the multi-layer perceptron operation function, Avgpool() represents the global average pooling operation function, and Maxpool() represents the maximum pooling operation function. Then, the obtained channel attention matrix M C Applied to the input feature map X (L) , generate enhanced feature map F′∈R C × S ×H× W : F′=M c (X (L) )⊙X (L) ); Among them, ⊙ represents element-wise multiplication.

8. The method for predicting the remaining service life of mechanical equipment based on 3Dformer according to claim 7, characterized in that: Step S4 is as follows: the prediction module based on sequence decomposition takes the enhanced feature map F′ as input, and uses the sequence decomposition layer I to obtain the seasonal component F from the enhanced feature map F′. s ′1; through the multi-head attention mechanism layer from the seasonal component F s ′1 to obtain the 3D tensor MultiHead (Q t ,K t ,V t );The seasonal component F is added through Add&Norm layer I s ′1 and 3D tensor MultiHead(Q t ,K t ,V t ) is subjected to residual connection and normalization processing to obtain normalized information I; the seasonal component F is extracted from the normalized information I through the sequence decomposition layer II s ′2, and then add the seasonal component F s ′1 and seasonal component F s ′2 is fused and normalized to obtain normalized information II, and feature conversion is performed through feedforward network II to obtain feature conversion result I; seasonal component F is converted through feedforward network I s ′2 performs feature conversion to obtain feature conversion result II; use Add&Norm layer III to convert seasonal component seasonal component F s ′2 and the feature conversion result II are residually connected and normalized to obtain the normalized information III; the feature conversion result I and the normalized information III are fused and normalized using the Add&Norm layer IV and then sent to the fully connected layer for integrated regression to output the RUL prediction value.

9. The method for predicting the remaining service life of mechanical equipment based on 3Dformer according to claim 8, characterized in that: The method for obtaining seasonal components is: using multiple filters of different sizes to extract multiple trend items from input data; aggregating the extracted multiple trend items into a final trend component through weights based on the correlation between the extracted multiple trend items and the input data; Obtain the seasonal component based on the trend component and the difference of the input data.

10. The method for predicting the remaining service life of mechanical equipment based on 3Dformer according to claim 9, characterized in that: The method for extracting multiple trend items from input data is as follows: Among them, d model Indicates the number of average pooling filters, Padding d () indicates the filling operation function, Avgpol d () represents the average pooling operation function, Concat represents the concatenation operation, F″(F) represents the extracted multiple trend items, and F represents the input data; The method of aggregating the final trend components through weights is: F t ′=Softmax(L(F))F″(F) Among them, L() represents the fully connected layer operation, Softmax() is the activation function, Softmax(L(F)) represents the weight of the trend item used for fusion extraction, and F t '' is the trend component; The calculation method of the seasonal component is: s ′=FF t ′.

Images