A method and device for predicting the remaining useful life of a mechanical equipment

Abstract

The application provides a mechanical equipment residual service life prediction method and device, and relates to the technical field of equipment management.The application adopts an adaptive pruning algorithm for light processing, and continuously optimizes model performance through error calculation and model evaluation, so that the prediction speed and prediction accuracy are improved; the pruning process automatically prunes redundant elements through an adaptive structured pruning strategy, so that unnecessary operations and searches are avoided; and when different neural network layers are used, different pruning rates are used for automatic structure pruning, so that an optimal light network model is finally obtained; the residual service life prediction accuracy is significantly improved, and less storage space is occupied, so that the application can be conveniently deployed on platforms such as small embedded systems, and timely maintenance and maintenance of maintenance personnel are reminded, so that the application is suitable for preventive maintenance and management of various mechanical equipment, and has good practicability and popularization value.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of equipment management, and in particular to a mechanical equipment remaining service life prediction method and device. BACKGROUND

[0002] With the development of modern industry, the service life of mechanical equipment has become an important factor restricting production efficiency and quality; accurately predicting the remaining life of mechanical equipment can effectively improve equipment utilization and reduce production costs. Predictive and health management (PHM) technology, as an emerging system reliability evaluation method, combines sensor performance parameter monitoring, data acquisition, fault diagnosis and life prediction methods to realize on-demand maintenance of equipment, and can provide early warning of faults, improve the safety and reliability of the system, and has been widely applied in various industrial fields and achieved remarkable results.

[0003] Most existing remaining life prediction methods for mechanical equipment use deep learning-based life prediction models; however, most mechanical equipment remaining service life prediction models have the problems of large parameter redundancy, poor model calculation efficiency and real-time performance. Too many parameters make the model redundant and heavy, making it difficult to achieve the best prediction accuracy, which not only wastes computing resources, but also causes serious over-parameterization and overfitting problems.

[0004] Therefore, under the premise of not affecting the model prediction performance, how to apply structured pruning technology to the field of mechanical equipment life prediction is a key problem. In addition, in the process of exploring model pruning methods to achieve model lightweight, if the same pruning rate is used for pruning all network layers of the model, it may cause the pruning rate of sensitive layers with high parameter importance to be too high, resulting in unnecessary accuracy loss. Therefore, how to find the optimal pruning rate for each layer of the network model to obtain the optimal lightweight life prediction model is also a key problem.

[0005] In view of the above, it is necessary to provide a mechanical equipment remaining service life prediction method and device to solve the above technical problems. SUMMARY

[0006] To solve the above technical problems, the present application provides a mechanical equipment remaining service life prediction method, which sequentially performs data preprocessing steps, lightweight model construction steps and remaining service life prediction steps, and uses a lightweight model to predict the future life of mechanical equipment; wherein,

[0007] Data preprocessing step: feature selection and construction of time series features according to the ability of input data to represent degradation information; normalization processing is performed on the selected data to remove noise; the data set for model training and testing is constructed by adding the remaining useful life label of segmented degradation and dividing the time step of input data;

[0008] Lightweight model construction step: the remaining useful life value corresponding to the next time of each group of time series is taken as the label of the model training stage; the model is trained through multiple iterations, and the model parameters are updated according to the training reflected by the loss function, so as to continuously reduce the model loss; an adaptive pruning method is used to perform a pruning process on different neural network layers to effectively delete redundant elements in the model, and an optimal lightweight network model is obtained; the Adam optimizer is used for model training and optimization, and the Dropout method is introduced to avoid overfitting; and a random search strategy is applied to obtain an optimal lightweight model in a large hyperparameter space, and a trained lightweight Transformer model is obtained;

[0009] Remaining useful life prediction step: input the test set into the trained lightweight Transformer model, test the model performance and draw the equipment performance degradation trend curve, and finally output the remaining useful life prediction value and prediction error of the mechanical equipment.

[0010] As a further solution, in the lightweight model construction step, only the encoder part of the Transformer model is used to complete the remaining useful life prediction of the mechanical equipment; and a position encoding operation is added to the input sequence of the encoder; by normalizing the position encoding, the position information is integrated into the input sequence; the position encoding matrix and the mechanical equipment feature matrix are added, so that the input of the Transformer model has position encoding; wherein,

[0011] p={1,2,…,n}

[0012]

[0013] E=reshape(PE)

[0014] In the formula, n is the total number of mechanical equipment features, min is the minimum value of p, and max is the maximum value of p; the reshape(·) function transforms PE into a matrix with the same size as the mechanical equipment feature matrix.

[0015] As a further solution, in the adaptive pruning process: by finding a suitable weight W, the size of the model has the smallest loss function L(D, W) under a limit C; wherein, the pruning optimization function is as follows:

[0016]

[0017] where s denotes the weights are divided into groups; w s is an element of the structure group S with weight W.

[0018] As a further solution, structured pruning simplification is performed: first, a W with a smaller Loss is trained on the dataset D; then the importance score of each weight group is evaluated, and the weight groups with low importance are pruned until the target constraint is reached; wherein the importance evaluation function of each group is as follows:

[0019] I S = (L(D, W|w s = 0) - L(D, W)) 2

[0020] To further simplify the calculation, the first-order Taylor expansion of the loss value L is used instead of the value of L at w s = 0:

[0021] L(D, W|w s = 0) = L(D, W) + ∑ s∈S g s w s

[0022] where, represents the gradient of the training target with respect to w s ;

[0023] Substitute L into the original importance function to obtain the importance function of the simplified weight groups as follows:

[0024] I S (W) = (∑ s∈S g s w s ) 2

[0025] After multiple iterations of training the weights, the training process continues until convergence; and in the fine-tuning process, the importance of each group of weights is evaluated by the importance function, and the unimportant weights are accurately pruned.

[0026] As a further solution, the optimal pruning rate is adaptively determined through reinforcement learning: the deep deterministic policy gradient DDPG algorithm based on offline policy is used to adaptively control the pruning rate; through the iterative learning of the agent, the optimal pruning rate of each layer of the network model can be found under the given network compression target; different pruning rates are used for automatic structural pruning for different neural network layers, and finally an optimal lightweight network model is obtained; wherein the DDPG algorithm consists of an Actor network and a Critic network.

[0027] As a further solution, the DDPG algorithm performs inference learning by the following steps: at the t-th time step, the agent receives a state S t from the environment and outputs an action A t as the compression rate of the current layer; the DDPG algorithm follows a predefined network pruning strategy to perform the current layer adaptive pruning process, and the network model is simplified to a new architecture; when the environment state is updated from S t to S t+1 , the short-term reward R t is fed back to the agent; for the next layer, the agent will receive the new environment state generated by the pruning of the previous layer, perform the same adaptive pruning process, until the predefined target network structure is reached; after completing the pruning of the last layer, the model will evaluate the accuracy of the pruned model on the validation set, and return the reward R FLOPs to the agent.

[0028] As a further solution, in order to improve the exploration ability of the agent, the Actor network performs direct mapping of each state to an action, defined as a deterministic policy; wherein the action is usually mixed with noise generated by truncated normal distribution, defined as follows:

[0029]

[0030] In each parameter update iteration, the maximum average reward value sampled from the buffer is used to update the Actor network, defined as follows:

[0031]

[0032] wherein the Critic network Q(S i , A i ) represents the Q value related to the state S i and the action A i .

[0033] As a further solution, in order to update the network, the current target value calculated by the Critic network is defined as:

[0034] y i = R i + γQ'(S i+1 , π'(S i+1 | θ π′ ) | θ Q′ )

[0035] wherein π'(S i | θ π′ ) is the target value of the Actor network; Q'(S i , Ai |θ Q′ ) is the target value of Critic network;

[0036] To update all parameters of Critic network, DDPG algorithm uses mean square error loss function and gradient back propagation to minimize the loss between estimated value and target network value, that is:

[0037]

[0038] Where Q(Si, Ai|θQ) is obtained by Critic network.

[0039] As a further solution, the pruning rate of each layer is learned using RL, based on the representation information of network structure; in order to evaluate different network structures, the performance measurement index of RL learning is defined as:

[0040]

[0041] In order to distinguish different sparse network models with similar model performance, the measurement index of RL learning efficiency is defined as:

[0042]

[0043]

[0044] Where C represents the FLOPs compression rate of pruned subnetwork compared with baseline network, FLOPs c represents the FLOPs value of pruned network, FLOPs b represents the FLOPs value of original network;

[0045] Combining the measurement indexes of model performance and efficiency, the score reflecting the good or bad of each sparse network model is obtained, and it is used as the reward function R c :

[0046]

[0047] Where finding the optimal sparse network structure is equivalent to searching for the architecture that obtains the highest measurement score R c .

[0048] A mechanical equipment residual useful life prediction device, comprising a data acquisition module, a model training module, an adaptive pruning algorithm module and an RUL prediction module; wherein,

[0049] The data acquisition module is used to collect historical operating data of mechanical equipment and convert it into sequential data through data preprocessing; the model training module is used to train the Transformer model using the sequential data; the adaptive pruning algorithm module is used to adaptively prune the Transformer model to obtain the optimal lightweight model; and the RUL prediction module is used to predict the future lifespan of mechanical equipment using the lightweight model.

[0050] Compared with related technologies, the method and device for predicting the remaining service life of mechanical equipment provided by the present invention have the following beneficial effects:

[0051] 1. Improve prediction accuracy: Using the Transformer model for remaining lifetime prediction has stronger sequence modeling and representation capabilities compared to traditional machine learning algorithms. It can better capture key features in the sequence, reduce information loss and confusion, and improve prediction accuracy.

[0052] 2. Reduce computational complexity: By using an adaptive pruning algorithm to prune the model layer by layer, the computational complexity of the model can be significantly reduced, thereby improving the computational efficiency of the model, making it suitable for practical engineering applications;

[0053] 3. Adapting to different data characteristics: The adaptive pruning algorithm automatically adjusts the pruning ratio and pruning time according to the characteristics of the dataset and the model, which can adapt to different data characteristics and improve the model's generalization ability and prediction accuracy;

[0054] 4. Visualization and analysis: The prediction module can visualize and analyze the prediction results, making it easier for users to make subsequent decisions and manage them. Attached Figure Description

[0055] Figure 1 A flowchart of a method for predicting the remaining service life of mechanical equipment provided by the present invention;

[0056] Figure 2 A flowchart of a method for pruning adaptive optimal lightweight models provided by the present invention;

[0057] Figure 3 A piecewise linear degradation model diagram provided by the present invention;

[0058] Figure 4 The graph shows the life prediction results for all engines in FD001-FD004 using the lightweight Transformer model.

[0059] Figure 5 Example diagrams showing the predicted remaining service life of individual engines in FD001-FD004;

[0060] Figure 6 The present invention provides a structural diagram of a device for predicting the remaining service life of mechanical equipment. Detailed Implementation

[0061] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0062] like Figure 1 As shown, this embodiment provides a method for predicting the remaining service life of mechanical equipment, which sequentially performs a data preprocessing step, a lightweight model construction step, and a remaining service life prediction step, using a lightweight model to predict the future lifespan of the mechanical equipment; wherein,

[0063] Data preprocessing steps: Select features and construct time series features according to the strength of the input data's ability to represent degradation information; normalize the selected data to remove noise; construct a dataset for model training and testing by adding segmented degradation remaining lifetime labels and dividing the input data into time steps;

[0064] The lightweight model construction steps are as follows: The remaining lifetime value corresponding to the next time step of each time series is used as the label for the model training phase; the model is trained through multiple iterations, and the model parameters are updated and optimized based on the training progress reflected by the loss function, thereby continuously reducing the model loss; an adaptive pruning method is used to perform pruning on different neural network layers to effectively remove redundant elements in the model, resulting in an optimal lightweight network model; the Adam optimizer is used for model training and optimization, and the Dropout method is introduced to avoid overfitting; and a random search strategy is applied to obtain the optimal lightweight model in a large hyperparameter space, resulting in a well-trained lightweight Transformer model.

[0065] The remaining useful life prediction steps are as follows: input the test set into the trained lightweight Transformer model, test the model performance and plot the equipment performance degradation trend curve, and finally output the predicted value of the remaining useful life of the mechanical equipment and the prediction error.

[0066] It should be noted that existing prediction models suffer from high parameter redundancy and poor real-time computational efficiency, resulting in low model performance. To address the issues of low model performance and limited applicability (such as small embedded systems) caused by over-parameterization, this embodiment employs a method for constructing a lightweight model based on pruning techniques. This results in a higher-performance lightweight model that is better suited to the demanding resource environment of the Internet of Things, significantly improving its usability and availability. The resulting optimal lightweight network model can be deployed at the edge or in embedded mobile systems, occupying less storage space. Furthermore, this invention utilizes the Transformer model, which can process sequential data and maintain dependencies in long sequences, thereby improving prediction accuracy.

[0067] As a further solution, during the lightweight model construction step, only the encoder portion of the Transformer model is used to predict the remaining service life of the mechanical equipment; position encoding is added to the encoder's input sequence; position information is integrated into the input sequence through normalized position encoding; the position encoding matrix and the mechanical equipment feature matrix are added together, thus ensuring that the Transformer model's input has position encoding; where,

[0068] p = {1, 2, ..., n}

[0069]

[0070] E = reshape(PE)

[0071] In the formula, n is the total number of mechanical equipment features, min is the minimum value of p, and max is the maximum value of p; the reshape(·) function transforms PE into a matrix with the same size as the mechanical equipment feature matrix.

[0072] It should be noted that this invention only requires the encoder part of the Transformer to predict the remaining service life of mechanical equipment. The encoder is responsible for encoding the input sequence and mapping it into an intermediate vector containing input feature information. Its core principle is the self-attention mechanism. The self-attention mechanism is a variant of the attention mechanism. It represents the correlation by calculating the similarity between feature vectors, reducing dependence on external information, thereby solving the problem of long-distance dependency capture and possessing stronger feature extraction performance. The purpose of the self-attention mechanism is to filter out a small amount of important information from the input mechanical equipment feature sequence, using weights to represent the importance of information, so that the model focuses on more important information. The self-attention mechanism uses scaled dot product attention to calculate the attention value of the feature matrix. First, the query matrix and key matrix are multiplied by a dot product and SoftMax normalized to calculate the weight coefficients, and then the value matrix is ​​weighted and summed according to the weight coefficients. As shown in the following formula:

[0073]

[0074] Q = X f W Q

[0075] K = X f W K

[0076] V = X f W V

[0077] In the formula: Q is the query matrix, K is the key matrix, and V is the value matrix. These three matrices are derived from the input feature matrix x. f Each with the corresponding weight matrix W Q W K W V Multiplying them together, we get d, which is the dimension of the key matrix. The softmax function is used to obtain the weights of the values.

[0078] Self-attention mechanisms allow models to focus on certain important information from input features, but a single attention mechanism can only learn relevant information within a single representation space. To synthesize the importance of information contained in the input sequence, this invention employs a multi-head self-attention mechanism that collectively focuses on information from different representation subspaces at different locations. The multi-head self-attention mechanism can be described as a concatenation of multiple self-attention mechanisms. It utilizes h self-attention heads to individually achieve self-attention on the input sequence, and then cascades and performs linear transformations on the multiple attention values ​​to obtain the final attention value, enabling modeling and representation of different constraints. The formula is shown below:

[0079] MultiHead(Q,K,V)=Concat(h1,...,h m )·W

[0080]

[0081] In the formula: Let Q, K, and V be the weight matrix for the i-th attention head, W be the multi-head attention weight matrix, m be the number of attention heads, and the Concat function be used to concatenate the output values ​​calculated by each attention head.

[0082] The forward data Transformer processing layer and the backward data Transformer processing layer are structurally identical, differing only in their input data. Each Transformer processing layer consists of multiple identical layers stacked together. Each identical layer contains two sub-layers: a multi-head self-attention mechanism layer and a one-dimensional convolutional layer. To address the issue that increasing network depth negatively impacts the accuracy of engine remaining service life prediction, residual connections are used between each sub-layer. Furthermore, layer normalization is applied at the end of each sub-layer to improve the training speed and generalization ability of the neural network, as shown in the following formula:

[0083] o = LayerNorm(x + Sublayer(x))

[0084] In the formula: Sublayer is the processing function inside the self-attention mechanism layer and the one-dimensional convolutional layer, and LayerNorm is the layer normalization processing function.

[0085] The Transformer's advantage stems from its pure attention mechanism, but this causes it to lose the ability to learn sequence positional information. Currently, most mainstream neural networks possess positional information encoding capabilities; for example, the search window used in CNNs contains data positional information, and the recurrent structure of LSTMs inherently includes data positional information. The Transformer neural network also incorporates positional information encoding in its design, adding a positional encoding message to each input data point.

[0086] Since the length of mechanical equipment feature data is fixed, in order to intuitively and simply represent the positional relationship between engine feature matrix data, this invention uses a normalized position encoding method, which adds position encoding operation to the input sequence of the encoder to integrate position information into the input sequence.

[0087] The feature matrix data of the mechanical equipment is processed by a Transformer layer, and then by a fully connected Argmax layer. This layer performs a nonlinear mapping on the processing results of the forward and backward feature matrices output by the Transformer layer to obtain the final prediction result of the remaining service life of the mechanical equipment.

[0088] As a further solution, in the adaptive pruning process: a suitable weight W is found such that the model size has the minimum loss function L(D, W) under a constraint C; where the pruning optimization function is as follows:

[0089]

[0090] Where s represents dividing the weights into multiple groups; w s It is an element with weight W in the structure group S.

[0091] It should be noted that although Transformer demonstrates excellent ability to extract feature relationships between input sequences, its self-attention mechanism inevitably involves exhaustive and redundant computations when processing long sequence data, which limits its direct application to the field of remaining useful life prediction.

[0092] To address this, this invention employs an adaptive structured pruning strategy to automatically trim redundant elements in the Transformer network model, thereby constructing a lightweight Transformer model. This method effectively avoids unnecessary operations and searches, achieving faster and better results in predicting the remaining service life of mechanical equipment.

[0093] As a further solution, structured pruning is performed for simplification: First, a weighted array W with a small loss is trained on dataset D; then, the importance score of each weighted array is evaluated, and weighted arrays with low importance are pruned until the target constraint is met; the importance evaluation function for each group is as follows:

[0094] I S =(L(D,W|w) s =0)-L(D,W)) 2

[0095] To further simplify the calculation, a first-order Taylor expansion of the loss value L is used to replace L in w. s The value at =0:

[0096] L(D,W|w s =0)=L(D,W)+∑ s∈S g s w s

[0097] in, Indicates the training objective relative to w s The gradient;

[0098] Substituting L into the original importance function, we obtain the simplified importance functions for each group of weights as follows:

[0099] I S (W)=(∑ s∈S g s w s ) 2

[0100] After multiple iterations of training the weights, the training process continues until convergence; and during the fine-tuning process, the importance of each group of weights is evaluated through an importance function, and unimportant weights are precisely pruned.

[0101] It's important to note that deep neural network models contain a large number of redundant neurons and weights, which have a very low impact on the final result. For example, Transformer models often have hundreds of millions of parameters. Their massive size and high latency prevent them from being deployed in scenarios with high real-time requirements and limited computing resources. Therefore, it's crucial to make the model smaller while maintaining its performance. This invention uses structured pruning techniques on Transformer models, and through methods such as... Figure 2 The steps shown can be followed to structurally prune unimportant weight groups in the model, such as attention heads, attention mechanisms, feed-forward network matrices, or layers. Pruning includes weight pruning, attention pruning, network layer pruning, and pruning of other parts. By removing these inherent structural groups, unnecessary weights in the Transformer are reduced, thereby reducing inference time.

[0102] As a further solution, Taylor importance scores, reflecting the contribution of weights to the loss value, are used as a global pruning criterion and can be compared across all weighted layers. This structured loss formula is general and can be applied to any overlapping weighted reassemblies, with the goal of finding the optimal substructure for pruning the Transformer network. However, manual pruning algorithms are time-consuming and labor-intensive, and the pruned network may not have an optimal substructure. Therefore, this invention proposes using a reinforcement learning pruning process to further investigate adaptive structured pruning to adaptively determine the optimal pruning rate.

[0103] Reinforcement Learning (RL) essentially models a problem as a Markov Decision Process (MDP). The purpose of this invention is to find a lightweight network architecture that can be viewed as a sequential decision problem. An MDP consists of a quintuple M = {S, A, P, R, γ}, where S is the state space of the RL; A is the action space of the RL; P is the state transition function of the RL, P(s) = γ. t+1 |s t a tLet represent the probability score from state to action at time t; R is the reward function of RL; γ is the reward discount factor of RL, which is set to 1 during pruning. The adaptive structured pruning proposed in this invention is an improvement on the layer-by-layer pruning strategy of network models. This method treats the network model as the environment of RL, the feature information of the network model as the state value, the compression rate of floating-point operations (FLOPs) at each layer of the network as the action value, and the compression rate and model performance metrics as the reward value. Since the action space is a continuous space, this invention uses the Deep Deterministic Policy Gradient (DDPG) algorithm based on offline strategies to adaptively control the pruning rate; through iterative learning of the agent, the optimal pruning rate for each layer of the network model can be found under a given network compression target; different pruning rates are used for different neural network layers for automated structural pruning, ultimately obtaining an optimal lightweight network model; the DDPG algorithm consists of an Actor network and a Critic network.

[0104] As a further solution, the DDPG algorithm performs inference learning through the following steps: At time step t, the agent receives state S from the environment. t And output action A in this state t As the compression ratio of the current layer; the DDPG algorithm follows a predefined network pruning strategy to perform an adaptive pruning process for the current layer, simplifying the network model into a new architecture; the environment state changes from S t Updated to S t+1 At that time, short-term reward R t Feedback is given to the agent; for the next layer, the agent receives the new environmental state generated by the pruning of the previous layer and performs the same adaptive pruning process until the predefined target network structure is reached; after completing the pruning of the last layer, the model evaluates the accuracy of the pruned model on the validation set and awards it a reward R. FLOPs Return it to the intelligent agent.

[0105] As a further solution to enhance the agent's exploratory capabilities, the Actor network executes a direct mapping from each state to a specific action, defined as a deterministic policy. This action is typically mixed with noise generated by a truncated normal distribution, defined by the following formula:

[0106]

[0107] In each parameter update iteration, the Actor network is updated by maximizing the average reward value sampled from the buffer, as defined below:

[0108]

[0109] Among them, the Critic network Q(S)i A i ) represents the state S i And Action A i The relevant Q value.

[0110] As a further solution, in order to update the network, the current target value calculated by the Critic network is defined as:

[0111] y i =R i +γQ′(S i+1 ,π′(S i+1 |θ π′ )|θ Q′ )

[0112] Wherein, π′(S i |θ π′ Q′(S) represents the target value of the Actor network; i A i |θ Q′ ) represents the target value of the Critic network;

[0113] To update all parameters of the Critic network, the DDPG algorithm uses the mean squared error loss function and gradient backpropagation to minimize the loss between the estimated and target network values, i.e.:

[0114]

[0115] Among them, Q(S) i A i |θ Q (Source: Critics Network)

[0116] As a further solution, RL is used to learn the pruning rate of each layer, based on the representation information of the network structure; to evaluate different network structures, the performance metric of RL learning is defined as:

[0117]

[0118] To differentiate between different sparse network models with similar performance, a metric for RL learning efficiency is defined as follows:

[0119]

[0120]

[0121] Where C represents the FLOPs compression ratio of the pruned subnetwork compared to the baseline network, FLOPs c This represents the FLOPs value of the pruned network. b This represents the FLOPs value of the original network;

[0122] By combining metrics of model performance and efficiency, a score reflecting the quality of each sparse network model is obtained, and this score is used as the reward function R. c :

[0123]

[0124] Finding the optimal sparse network structure is equivalent to searching for the highest metric score R. c The architecture.

[0125] like Figure 6 As shown, a device for predicting the remaining useful life of mechanical equipment includes a data acquisition module, a model training module, an adaptive pruning algorithm module, and a RUL prediction module; wherein,

[0126] The data acquisition module is used to collect historical operating data of mechanical equipment and convert it into sequential data through data preprocessing; the model training module is used to train the Transformer model using the sequential data; the adaptive pruning algorithm module is used to adaptively prune the Transformer model to obtain the optimal lightweight model; and the RUL prediction module is used to predict the future lifespan of mechanical equipment using the lightweight model.

[0127] It should be noted that the data acquisition module can obtain historical operating data of mechanical equipment, such as parameters like temperature, pressure, vibration, and current, through sensors and data acquisition devices. The collected data needs to undergo preprocessing, including data cleaning, missing value handling, and data transformation, to ensure data reliability and integrity. After preprocessing, the data is converted into sequential data for use in training the Transformer model and predicting remaining lifespan.

[0128] The model training module employs adaptive learning rate adjustment and early stopping strategies to avoid overfitting and improve the model's generalization ability. Lightweight model training utilizes a lightweight model obtained through adaptive pruning. During training, mini-batch stochastic gradient descent is used, combined with regularization and other techniques for optimization to enhance the model's generalization ability and prediction accuracy.

[0129] The adaptive pruning algorithm module, based on the characteristics of the model and the dataset, prunes the model layer by layer, removing unimportant neurons and connections to obtain the optimal lightweight model. The pruning ratio and pruning time are adaptively adjusted according to the current model's accuracy and computational complexity to ensure the lightweight model's prediction accuracy and computational efficiency.

[0130] The RUL prediction module uses a pre-trained lightweight model to predict the future lifespan of mechanical equipment. It takes historical data as input and outputs the predicted remaining lifespan over a future period. Furthermore, it can visualize and analyze the prediction results, including remaining life curves and anomaly detection results, facilitating subsequent decision-making and management for users.

[0131] In summary, this invention provides a method and apparatus for predicting the remaining service life of mechanical equipment based on a pruning adaptive optimal lightweight Transformer model. This method boasts high prediction accuracy and broad application prospects, playing a crucial role in the operation and maintenance of mechanical equipment. Analysis of real-world datasets demonstrates that the method not only finds the optimal balance between model prediction accuracy and pruning rate but also accurately and reliably predicts the remaining service life of mechanical equipment. Verification is as follows:

[0132] In one specific embodiment, the C-MAPSS aero-engine simulation condition monitoring dataset from the PHM08 International Conference on Fault Prediction and Health Management is used, and the implementation steps of the invention method are described.

[0133] This dataset is generally used as a benchmark dataset in the field of remaining service life prediction. The main components of an engine include: fan, combustion chamber, low-pressure rotor, low-pressure turbine, low-pressure compressor, high-pressure compressor, high-pressure rotor, high-pressure turbine, and exhaust nozzle. It contains four subsets, FD001 to FD004, under different operating conditions and failure modes. Each subset contains a training set, a test set, and remaining service life data. The training set contains multiple state parameter data for each time series within the complete cycle of the aero-engine from its initial state to complete failure. The test set contains the state parameters and corresponding remaining service life for a certain period before engine failure. The datasets used in the embodiments of this invention are shown in Table 1.

[0134] Table 1 Description of the C-MAPSS dataset

[0135]

[0136] 1) Feature selection

[0137] This invention uses the entire engine dataset for validation. Analysis of data from different sensors reveals that not all sensor measurements provide useful degradation information. To select sensor features that characterize significant engine degradation trends as input samples, this invention uses data with strong monotonicity and correlation to reflect the increasing or decreasing trends of these features. The correlation between the data and the remaining service life prediction is positively correlated with two indicators. Reasonable sensor data is selected by calculating a linear combination of these two indicators, as follows:

[0138]

[0139]

[0140] In the formula, K is the time series length of the entire life cycle, and x i and y i These represent the feature and time values, respectively. Mono = 1 indicates a strong monotonic trend in the feature, and Mono = 0 indicates a non-monotonic trend in the feature. Corr = -1 indicates that the feature value is strictly decreasing, and Corr = 1 indicates that the feature value is strictly increasing.

[0141] 2) Normalization

[0142] Monitoring data returned by multiple sensors in an engine often have different dimensions and units. To eliminate the impact of non-standardized data on the prediction of remaining service life, this invention performs overall normalization on the raw data to improve the model's convergence speed and prediction accuracy. The specific formula is as follows:

[0143]

[0144] In the formula, This is the raw monitoring data, where j represents the j-th sensor and i represents the i-th data point. and Let be the minimum and maximum values ​​of the j-th feature, respectively. It is the normalized data.

[0145] 3) Remaining service life segment labels

[0146] The remaining service life label value has a significant impact on the prediction results. Engine performance monitoring variables show good performance in the initial stage of operation. To improve the accuracy of model predictions, this invention ignores engine degradation during this stage. After a period of operation, the engine's health condition continuously declines, and the failure rate increases rapidly with time, eventually reaching a severe failure state. In the linear degradation stage, the remaining service life threshold is set to 125 cycles, based on the average lifespan of the aero-engine. When it is greater than or equal to this threshold, the remaining service life is a constant value; when it is less than the threshold, the remaining service life becomes the actual value and decreases linearly with the cycle. The piecewise linear degradation model is as follows: Figure 3 As shown.

[0147] In the remaining useful life prediction process, the remaining useful life is used as the target output based on a set of data. The remaining useful life in the training set is calculated as follows:

[0148] RUL train =cycle max -cycle

[0149] RUL test =cycle max +RUL-cycle

[0150] In the formula, cycle max The maximum number of engine operating cycles is given by 'cycle', where 'cycle' is the current number of engine operating cycles, and 'remaining service life' is the actual remaining service life.

[0151] Lifespan prediction effect

[0152] This invention innovatively incorporates an adaptive structured pruning mechanism into the Transformer network. To verify the effectiveness of the proposed prediction model, multiple comparative experiments were conducted, selecting commonly used CNNs, RNNs, LSTMs, and the traditional Transformer as comparison models. The comparative experiments show that this method significantly outperforms other methods with lower RMSE and scores. As shown in Table 2:

[0153] Table 2 Comparison of the model in this invention with other popular models

[0154]

[0155] like Figure 4 The lightweight Transformer model shown provides life prediction results for all engines in FD001-FD004, and as follows: Figure 5 The examples shown are predictions of the remaining service life of individual engines in FD001-FD004. Compared to the original Transformer model, after using the lightweight Transformer to predict the remaining service life, the RMSE and score of the FD001 data decreased by approximately 7% and 5%, respectively; the RMSE and score of the FD002 data decreased by approximately 7% and 22%, respectively; the RMSE and score of the FD003 data decreased by approximately 4% and 2%, respectively; and the RMSE and score of the FD004 data decreased even more significantly, by approximately 6% and 24%. The main reason is that the addition of the adaptive structured pruning mechanism effectively reduced the redundancy of the model and could find more accurate data degradation trends from the sequence information extracted from the original Transformer network, effectively improving the accuracy of remaining service life prediction. This indicates that it can accurately predict the remaining operating life of equipment in practical use, thus greatly reducing unnecessary equipment maintenance, saving maintenance costs, and preventing safety accidents.

[0156] In practical engineering applications, this invention can be applied to predict the remaining life of various mechanical equipment, such as engines, bearings, gearboxes, and wind turbines. Furthermore, this invention can be combined with technologies such as the Industrial Internet and big data to achieve remote monitoring and intelligent maintenance of mechanical equipment, improving equipment stability and reliability, and reducing maintenance costs and production downtime.

[0157] The above are merely embodiments of the present invention and do not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for mechanical equipment remaining useful life prediction, characterized in that, The data preprocessing step, the lightweight model construction step and the remaining useful life prediction step are sequentially executed, and the lightweight model is used to predict the future life of the mechanical equipment; wherein, The data preprocessing step: selecting features according to the ability of input data to represent degradation information and constructing time series features; normalizing the selected data to remove noise; adding a segmented degradation remaining useful life label and dividing the time step of the input data to construct a data set for model training and testing; the data preprocessed is the historical operation data of the mechanical equipment, including temperature, pressure, vibration, current parameters; The lightweight model construction step: taking the next time point of each group of time series as the label of the model training stage; the model is trained through multiple iterations, and the model parameters are updated and optimized according to the training reflected by the loss function, so as to continuously reduce the model loss; an adaptive pruning method is used to perform a pruning process on different neural network layers to effectively delete redundant elements in the model, and an optimal lightweight network model is obtained; the Adam optimizer is used for model training and optimization, and the Dropout method is introduced to avoid overfitting; and a random search strategy is applied to obtain an optimal lightweight model in a large hyperparameter space, and a trained lightweight Transformer model is obtained; The remaining useful life prediction step: inputting the test set into the trained lightweight Transformer model, testing the model performance and drawing the equipment performance degradation trend curve, and finally outputting the remaining useful life prediction value and prediction error of the mechanical equipment; When performing the lightweight model construction step, only the encoder part of the Transformer model is used to complete the remaining useful life prediction of the mechanical equipment; and a position encoding operation is added to the input sequence of the encoder; the position information is integrated into the input sequence by normalizing the position encoding; the position encoding matrix and the mechanical equipment feature matrix are added, so that the input of the Transformer model has position encoding; wherein, P = {1, 2, …, n} E = reshape(PE) In the formula, n is the total number of mechanical equipment features, min is the minimum value of p, and max is the maximum value of p; the reshape(·) function transforms PE into a matrix with the same size as the mechanical equipment feature matrix; In the adaptive pruning process: find a suitable weight W, so that the size of the model has the smallest loss function L(D, W) under a certain constraint C; Simplify the structured pruning: first, train W on the data set D to get a smaller loss; then evaluate the importance score of each weight group, and prune the weight group with low importance until the target constraint is reached.

2. The method for predicting the remaining useful life of mechanical equipment according to claim 1, wherein, The pruning optimization function is as follows: min W L(D, W), s.t.‖∑ s∈S |w s |‖0≤C where s denotes the division of the weights into groups; w s is an element of the structure group S with weight W.

3. A method for predicting the remaining useful life of a mechanical plant according to claim 2, characterized in that, The importance evaluation function of each group is as follows: I S = (L(D, W | w s = 0) - L(D, W)) 2 To further simplify the calculation, a first order Taylor expansion of the loss value L is used instead of L at w s = 0: L(D, W | w s = 0) = L(D, W) + ∑ s∈S g s w s wherein, represents the gradient of the training objective with respect to w s ; Substitute L into the original importance function to obtain the importance function of each group of weights after simplification as follows: I S (W) = (∑ s∈S g s w s ) 2 After multiple iterations of training weights, the training process continues until convergence; and in the fine-tuning process, the importance of each group of weights is evaluated by the importance function, and the unimportant weights are accurately pruned.

4. The method for predicting the remaining useful life of mechanical equipment according to claim 3, characterized in that, Adaptive determination of optimal pruning rate through reinforcement learning: a deep deterministic policy gradient (DDPG) algorithm based on an offline policy is used to adaptively control the pruning rate; through iterative learning of the agent, the optimal pruning rate for each layer of the network model can be found under a given network compression target; different pruning rates are used for automatic structural pruning of different neural network layers, and finally an optimal lightweight network model is obtained; the DDPG algorithm consists of an actor network and a critic network.

5. A method for predicting the remaining useful life of a mechanical plant according to claim 4, characterized in that, The DDPG algorithm performs inference learning by the following steps: at the t-th time step, the agent receives a state S t from the environment and outputs an action A t as the current layer's compression rate; The DDPG algorithm follows a predefined network pruning strategy to perform a current layer adaptive pruning process, the network model is simplified to a new architecture; the environment state from S t is updated to S t+1 , the short-term reward R t is fed back to the agent; for the next layer, the agent will receive the new environment state generated by the pruning of the previous layer, perform the same adaptive pruning process, until the predefined target network structure is reached; after completing the pruning of the last layer, the model will evaluate the accuracy of the pruned model on the validation set, and return the reward R FLOPs to the agent.

6. A method for predicting the remaining useful life of mechanical equipment according to claim 5, wherein, To improve the exploration ability of the agent, the actor network performs direct mapping of each state to an action, defined as a deterministic policy; the action is usually mixed with noise generated by a truncated normal distribution, defined as follows: In each parameter update iteration, the maximum average reward value sampled from the buffer is used to update the actor network, defined as follows: wherein the Critic network Q(S i , A i ) represents a Q-value associated with state S i and action A i .

7. A method for predicting the remaining useful life of a mechanical plant according to claim 6, characterized in that, To update the network, the current target value calculated by the critic network is defined as: y i = R i + γQ'(S i+1 , π'(S i+1 |θ π′ )|θ Q′ ) wherein π'(S i |θ π′ ) is the target value of the Actor network; Q'(S i , A i |θ Q′ ) is the target value of the Critic network; To update all parameters of the critic network, the DDPG algorithm uses the mean square error loss function and gradient backpropagation to minimize the loss between the estimated value and the target network value, i.e. where Q(S i , A i |θ Q ) is obtained from a network of critics.

8. A method for predicting the remaining useful life of mechanical equipment according to claim 7, characterized in that, RL is used to learn the pruning rate of each layer based on the representation information of the network structure; to evaluate different network structures, the performance metric of RL learning is defined as: To distinguish different sparse network models with similar model performance, the performance metric of RL learning efficiency is defined as: Wherein, C represents the FLOPs compression rate of the pruned subnetwork compared with the baseline network, FLOPs c represents the FLOPs value of the pruned network, FLOPs b represents the FLOPs value of the original network; In combination with the performance and efficiency of the model, the score reflecting the good and bad of each sparse network model is obtained, and is used as a reward function R c : where finding the optimal sparse network structure is equivalent to searching for the highest metric score R c architecture.

9. A device for predicting the remaining useful life of mechanical equipment, for implementing the method for predicting the remaining useful life of mechanical equipment according to any one of claims 1-8, characterized in that, It includes a data acquisition module, a model training module, an adaptive pruning algorithm module and an RUL prediction module; wherein, The data acquisition module is used to collect historical operation data of mechanical equipment, including temperature, pressure, vibration, current parameters, and convert them into sequence data through data preprocessing; the model training module is used to train the Transformer model using sequence data; the adaptive pruning algorithm module is used to adaptively prune the Transformer model to obtain an optimal lightweight model; the RUL prediction module is used to predict the future life of the mechanical equipment using the lightweight model.

Images