Predictive maintenance method for mechanical equipment

Abstract

The invention discloses a predictive maintenance method for mechanical equipment, and relates to the technical field of aero-engine maintenance, and the method comprises the steps: obtaining the state data of the mechanical equipment at a plurality of moments, inputting the state data into a state conversion network model, extracting the multi-sensory characteristics of the state data at different scales through the state data, and obtaining the state conversion network model; obtaining a belief state; the belief state is the feature representation of the state data; and inputting the belief state into a pre-trained real-time conversion-near-end strategy optimization network model to obtain a predictive maintenance result of the mechanical equipment. The method can reduce the complexity of predictive maintenance of the mechanical equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of aircraft engine maintenance, and in particular to a predictive maintenance method for mechanical equipment driven by a real-time sensor. Background Art

[0002] At present, the application scope of prognostics health management (PHM) is very wide, and it can be applied to aerospace, military, industrial manufacturing, energy, transportation and other fields. For example, for a Boeing 787, there are about 1,000 sensors recording the real-time operating parameters of the engine. These parameters indicate the potential status of the flight equipment. These data are the basis for the remaining useful life (RUL) prediction and predictive aircraft maintenance plan. In the field of aerospace, PHM can help aircraft and spacecraft monitor their health status in real time during flight, predict potential failures, and take corresponding measures in time to ensure the safety and reliability of flight. However, the purpose of predicting the RUL of mechanical equipment is to provide a scientific basis for the dynamic real-time maintenance of mechanical equipment. The simple RUL algorithm is incomplete because it lacks a maintenance plan. After obtaining the RUL of each state of the mechanical equipment, the staff needs to calculate the optimal ordering time and maintenance time. Such manual operation will undoubtedly cause meaningless cost losses. PHM should integrate the two, so that the health status of the equipment can be detected in real time or near real time, potential failures and RUL can be predicted, and the maintenance cost and failure cost can be combined for real-time maintenance and real-time decision-making of the system.

[0003] Currently, only a few studies have integrated data-driven RUL prediction into maintenance plans. However, there are still deficiencies in the method that combines real-time ordering with real-time maintenance, and the application of existing models is too cumbersome. Because existing models often retain all RULs, but the data of mechanical equipment monitored in long-term operation is a long time series, the early operating performance of the equipment is relatively stable, and the performance degradation is not obvious. The RUL of mechanical equipment decreases with the increase of service time, especially in the early service life. In order to obtain an accurate remaining service life, extremely complex algorithms are required, which makes the predictive maintenance of mechanical equipment more complicated. Summary of the invention

[0004] Based on this, it is necessary to provide a predictive maintenance method for mechanical equipment in response to the above technical problems, which method can reduce the complexity of predictive maintenance of mechanical equipment.

[0005] The present invention provides a predictive maintenance method for mechanical equipment, comprising:

[0006] Obtain status data of mechanical equipment at multiple times;

[0007] The state data is input into the state transition network model, and the multi-sensory features of the state data at different scales are extracted through the state transition network model to obtain the belief state; the belief state is the feature representation of the state data;

[0008] The belief state is input into the pre-trained real-time conversion-proximal strategy optimization network model to obtain the predictive maintenance results of mechanical equipment.

[0009] Preferably, obtaining the status data of the mechanical equipment at multiple times includes:

[0010] Obtaining the original state data collected in real time by multiple sensors on the mechanical equipment at multiple times;

[0011] The original state data is normalized to obtain the state data of the mechanical equipment at multiple moments.

[0012] Preferably, the state transition network model includes a convolutional neural network, a residual neural network, and a self-attention mechanism module. The state transition network model is used to extract multi-sensory features of different scales of state data to obtain a belief state, including:

[0013] Input the state data into the convolutional neural network to obtain the high-order features of the state data;

[0014] Inputting high-order features of the state data into the residual neural network to obtain high-order optimized features of the state data;

[0015] Input the high-order optimized features of the state data into the self-attention mechanism module to obtain the high-order belief state;

[0016] The high-order belief state of the state data is input into the linear neural network to obtain the belief state.

[0017] Preferably, the state transition network model further includes a bidirectional gated recurrent unit, and the method further includes:

[0018] The belief state is input into the bidirectional gated recurrent unit to obtain the remaining service life of the mechanical equipment.

[0019] Preferably, the real-time conversion-proximal strategy optimization network model includes a real-time ordering framework and a real-time maintenance framework; the belief state is input into the pre-trained real-time conversion-proximal strategy optimization network model to obtain the predictive maintenance results of the mechanical equipment, including:

[0020] Input the belief state into the real-time ordering framework to obtain the ordering strategy of the mechanical equipment, and determine the ordering strategy as the predictive ordering result;

[0021] The belief state is input into the real-time maintenance framework to obtain the maintenance strategy of the mechanical equipment, and the maintenance strategy is determined as the predictive maintenance result.

[0022] Preferably, the construction process of the real-time conversion-proximal strategy optimization network model includes:

[0023] Obtain sample belief states of sample mechanical devices;

[0024] Input the sample belief state into the policy network to obtain the sample predictive maintenance results of the sample mechanical equipment;

[0025] Input the current state and sample predictive maintenance results into the value network, and determine the evaluation value through the reward function in the value network;

[0026] Determine the strategy loss based on the sample predictive maintenance results and the sample real maintenance results, and determine the value loss based on the evaluation value;

[0027] Update the parameters of the policy network according to the policy loss, and update the parameters of the value network according to the value loss;

[0028] The updated policy network and value network are determined as the real-time transformation-proximal policy optimization network model.

[0029] Preferably, the sample belief state is input into the policy network to obtain a sample predictive maintenance result of the sample mechanical equipment, including:

[0030] Input the sample belief state into the policy network to obtain the action data of the sample mechanical equipment; the action data is the probability value of each behavior of the sample mechanical equipment; the behavior includes continued use, maintenance and ordering;

[0031] According to the probability value of each behavior, the sample predictive maintenance results are determined through a random strategy.

[0032] Preferably, the method further comprises:

[0033] Get the test belief status of the test mechanical equipment;

[0034] The test belief state is input into the real-time conversion-proximal strategy optimization network model to obtain the action data of the test mechanical equipment; the action data is the probability value of each behavior of the test mechanical equipment; the behavior includes early warning ordering, shutdown maintenance and continued operation;

[0035] Determine the predictive maintenance results of the test samples through randomization strategy;

[0036] Determine the true maintenance results of the test samples through a deterministic strategy;

[0037] According to the predictive maintenance results of the test samples and the actual maintenance results of the test samples, the performance evaluation results of the real-time conversion-proximal strategy optimization network model are determined.

[0038] Preferably, the real-time conversion-proximal policy optimization network model includes a real-time ordering framework, a reward function, including:

[0039]

[0040] Among them, r t is the evaluation value, t is the service life of the sample mechanical equipment, a t is the sample predictive maintenance result of the sample mechanical equipment, C is the total ordering cost of the sample mechanical equipment, T is the life of the mechanical equipment, L is the advance ordering period, SST is the status window, Order means warning ordering, and Continue means continued operation.

[0041] Preferably, the real-time conversion-proximal policy optimization network model includes a real-time maintenance framework, a reward function, including:

[0042]

[0043] Among them, r t is the evaluation value, t is the service life of the sample mechanical equipment, a t is the sample predictive maintenance result of the sample mechanical equipment, cr is the replacement cost of the sample mechanical equipment, cf is the replacement cost, SST is the status window, Replace means warning maintenance, and Continue means continued operation.

[0044] The present invention provides a predictive maintenance device for mechanical equipment, comprising:

[0045] An acquisition module is used to acquire status data of mechanical equipment at multiple times;

[0046] An extraction module is used to input the state data into the state transition network model, extract the multi-sensory features of the state data at different scales through the state data, and obtain the belief state; the belief state is the feature representation of the state data;

[0047] The prediction module is used to input the belief state into the pre-trained real-time conversion-proximal strategy optimization network model to obtain the predictive maintenance results of mechanical equipment.

[0048] The present invention provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the predictive maintenance method of the mechanical equipment is implemented.

[0049] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the predictive maintenance method of the mechanical equipment when executing the program.

[0050] At least one of the above technical solutions adopted by the present invention can achieve the following beneficial effects:

[0051] By extracting multi-sensory features of different scales through the state transition network model, it is possible to obtain rich feature representations of mechanical equipment status data at a low computational time cost. Such feature extraction can reduce the amount of raw data required for subsequent processing and reduce the computational load. Based on effective feature extraction, the input dimension of the real-time conversion-proximal strategy optimization network model is reduced, and the predictive maintenance results are determined directly through the pre-trained real-time conversion-proximal strategy optimization network model, which reduces the complexity of the algorithm as a whole, making the predictive maintenance process simpler. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0053] Figure 1 A schematic diagram of a process flow of a predictive maintenance method for mechanical equipment provided by the present invention;

[0054] Figure 2 A state transition network framework model diagram provided by the present invention;

[0055] Figure 3 It is a flow chart of another predictive maintenance method for mechanical equipment provided by the present invention;

[0056] Figure 4 It is a simulation of the schematic diagram of an aircraft engine;

[0057] Figure 5 A schematic flow chart of another method for predictive maintenance of mechanical equipment provided by the present invention;

[0058] Figure 6 It is a structural block diagram of a predictive maintenance device for mechanical equipment provided by the present invention;

[0059] Figure 7 A schematic diagram of computer equipment for a predictive maintenance method for mechanical equipment provided by the present invention.

[0060] Description of reference numerals:

[0061] 1. Fan; 2. Combustion chamber; 3. Low-pressure rotor speed; 4. Low-pressure turbine; 5. Low-pressure compressor; 6. High-pressure compressor; 7. High-pressure rotor speed; 8. High-pressure turbine; 9. Nozzle. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] The predictive maintenance method for mechanical equipment provided by the present invention can be executed by a server, which can be a server set on a business platform, or a device such as a desktop computer, a laptop computer, etc. that can execute the solution of the present invention. For the convenience of description, the following description is based on the server as the execution subject.

[0064] Figure 1 The following is a flow chart of a predictive maintenance method for mechanical equipment in this specification, which specifically includes the following steps:

[0065] S101, obtaining status data of mechanical equipment at multiple moments.

[0066] Mechanical equipment can be aircraft engines, military equipment, industrial manufacturing equipment, transportation equipment, energy equipment, etc.

[0067] In an exemplary embodiment, obtaining state data of a mechanical device at multiple moments includes: obtaining original state data collected in real time by multiple sensors on the mechanical device at multiple moments, and normalizing the original state data to obtain the state data of the mechanical device at multiple moments.

[0068] The mechanical equipment includes multiple sensors, which are used to collect the status data of the mechanical equipment in real time. When predictive maintenance of the mechanical equipment is required, the server can obtain the original status data collected by the sensor at multiple times in real time from the sensor. The original status data can be expressed as, X = [x1, x2, ..., x LD ] T , Where NC represents the number of sensors, and LD represents the length of the sensor data sequence. Represents the i-th sensor data in the k-th time period.

[0069] The original state data at each moment is input into a normalization algorithm, and the original state data at each moment is normalized by the normalization algorithm to obtain state data at multiple moments. The normalization algorithm can be processed using a Min-max scaling method.

[0070] S102, inputting the state data into the state transition network model, extracting multi-sensory features of different scales of the state transition network model, and obtaining the belief state; the belief state is a feature representation of the state data.

[0071] Multi-sensory features are data obtained by multiple sensors in multiple dimensions, which are used to represent the characteristics of the system in multiple dimensions. For aircraft engines, multiple sensors are distributed at different locations of the engine to collect data at different locations of the engine. The data of multiple sensors are integrated together to represent the characteristics of the engine in multiple dimensions. The belief state is a probabilistic representation of the state of the environment in which the agent is located. The belief state integrates all past observation information and prior knowledge of environmental dynamics, and is used to estimate the probability distribution of various real states that the environment may be in. It is a data representation after removing the noise from the original data. For aircraft engines, the raw data of the sensor contains a lot of noise, and the potential degradation state of the engine cannot be directly observed. The probability distribution of the potential state of the engine can be estimated, and this probability distribution is the belief state of the engine.

[0072] like Figure 2 As shown, Figure 2 : is a structural diagram of a state transition network model, and the state transition network model includes a convolutional neural network, a residual neural network, a self-attention mechanism module, and a linear neural network module. Therefore, in an exemplary embodiment, the multi-sensory features of different scales of the state data are extracted through the state transition network model to obtain the belief state, including: inputting the state data into the convolutional neural network to obtain the high-order features of the state data; and inputting the high-order features of the state data into the residual neural network to obtain the high-order optimization features of the state data; and then inputting the high-order optimization features of the state data into the self-attention mechanism module to obtain the high-order belief state, and finally, inputting the high-order belief state of the state data into the linear neural network to obtain the belief state. Among them, the state transition network model also includes a bidirectional gated recurrent unit, and the belief state is input into the bidirectional gated recurrent unit to obtain the remaining service life of the mechanical equipment.

[0073] In an exemplary embodiment, the self-attention mechanism module is a deep autoencoder with an attention strategy for selectively extracting multisensory features of different scales, and the bidirectional gated recurrent unit module is used to comprehensively extract degradation features of different time scales. The state transition network model fuses the degradation features and predicts the RUL.

[0074] During the state transition model training process, the mean squared error (MSE) loss function is constructed, the loss is calculated and the error is back-propagated, the neural network parameters of the state transition network model (TransStateNet) are updated, and the belief state of TransStateNet is used as the input of the real-time transition-proximal strategy optimization network space. The belief state will maintain the same size as the original mechanical equipment data, that is, ζ=x1,x2,…,x LD ] T ,

[0075] TransStateNet is constructed to transform the initial state of the original mechanical equipment. Compared with the original data, TransStateNet eliminates the influence of redundant features by extracting the degradation features of multi-sensor data at different time scales, and reduces the difficulty of reinforcement learning to fit the state distribution. The belief state of TransStateNet will be input into the real-time maintenance or real-time decision-making framework. This framework decides in real time whether to continue operation, issue an early warning, or shut down for maintenance based on the processed state. The pseudo code of TransStateNet is as follows:

[0076]

[0077]

[0078] S103, inputting the belief state into a pre-trained real-time conversion-proximal strategy optimization network model to obtain a predictive maintenance result of the mechanical equipment.

[0079] The results of predictive maintenance of mechanical equipment include early warning ordering, shutdown maintenance and continued operation.

[0080] The real-time conversion-proximal strategy optimization network model includes a real-time ordering framework and a real-time maintenance framework. The belief state is input into the real-time ordering framework to obtain the ordering strategy of the mechanical equipment, and the ordering strategy is determined as a predictive ordering result; the belief state is input into the real-time maintenance framework to obtain the maintenance strategy of the mechanical equipment, and the maintenance strategy is determined as a predictive maintenance result.

[0081] The results of predictive ordering include early warning ordering and continued operation, and the results of predictive maintenance include downtime maintenance and continued operation.

[0082] In an exemplary embodiment, the construction process of the real-time conversion-proximal policy optimization network model includes: obtaining the sample belief state of the sample mechanical equipment, and inputting the sample belief state into the initial real-time conversion-proximal policy optimization network to obtain the sample predictive maintenance result of the sample mechanical equipment; inputting the current state and the sample predictive maintenance result into the value network, and determining the evaluation value through the reward function in the value network; determining the strategy loss according to the sample predictive maintenance result and the sample actual maintenance result, and determining the value loss according to the evaluation value; updating the parameters of the initial proximal policy network according to the strategy loss, and updating the parameters of the value network according to the value loss; determining the updated initial proximal policy network model as the real-time conversion-proximal policy optimization network model.

[0083] Among them, before constructing the real-time conversion-proximal strategy optimization network model, the sample original state data collected in real time by multiple sensors on the sample mechanical equipment at multiple times can be obtained first, and then the sample original state data can be normalized to obtain the sample belief state of the sample mechanical equipment; the method of normalizing the sample original state data is the same as the method of normalizing the original state data in the above embodiment, and this embodiment will not be repeated here.

[0084] Optionally, the sample belief state is input into the initial proximal strategy optimization network to obtain a sample predictive maintenance result of the sample mechanical equipment, including: inputting the sample belief state into the initial proximal strategy network to obtain the action data of the sample mechanical equipment; the action data is the probability value of each behavior of the sample mechanical equipment; the behavior includes early warning ordering, shutdown maintenance and continued operation; according to the probability value of each behavior, the sample predictive maintenance result is determined by a random strategy.

[0085] The sample belief state is input into the initial proximal policy network to obtain the motion data of the sample mechanical equipment output by the initial proximal policy network.

[0086] According to the probability value of each behavior, the sample predictive maintenance result is determined through a random strategy, including: randomly obtaining a behavior from all behaviors as the sample predictive maintenance result; for example, randomly continuing to run as the sample predictive maintenance result of the sample mechanical equipment.

[0087] The proximal policy optimization network includes a real-time ordering framework and a real-time maintenance framework. Therefore, inputting the sample belief state into the initial proximal policy network to obtain the sample predictive maintenance result of the sample mechanical equipment, that is, inputting the sample belief state into the real-time ordering framework to obtain the sample predictive maintenance result of the sample mechanical equipment, and this sample predictive maintenance result includes continuing to operate and warning for ordering; inputting the sample belief state into the real-time maintenance framework to obtain the sample predictive maintenance result of the sample mechanical equipment, and this sample predictive maintenance result includes continuing to operate and warning for maintenance.

[0088] Both the real-time ordering framework and the real-time maintenance framework include a value network. Inputting the sample current state and the predictive maintenance result into the value network, and through the reward function in the value network, the evaluation value is determined, including: inputting the sample current state and the predictive maintenance result into the value network of the real-time ordering framework, and through the reward function in the value network of the real-time ordering framework, the evaluation value of the real-time ordering framework is determined; inputting the current state and the sample predictive maintenance result into the value network of the real-time maintenance framework, and through the reward function in the value network of the real-time maintenance framework, the evaluation value of the real-time maintenance framework is determined; the evaluation value represents the value of adopting the sample predictive maintenance result under the current state.

[0089] In an exemplary embodiment, the construction process of the real-time ordering framework is as follows:

[0090] If the framework issues an order demand at t = T - L, the order arrives when the engine remaining useful life RUL = 0. At this time, there is no holding cost and shortage cost, and the total cost C is the spare part ordering price c.

[0091] If the framework issues an order demand at t < T - L, the order arrives when the engine remaining useful life RUL > 0. At this time, there is a holding cost, and the total cost can be expressed as:

[0092] C = c + c·I·(T - L - t) (1)

[0093] Where, I represents the rate of return on funds, t is the service time, T is the life of the mechanical equipment, and L is the order lead time.

[0094] If the framework issues an order demand at t > T - L and t <= T, the order arrives when the engine remaining useful life RUL < 0. At this time, there is a shortage cost, and the total cost can be expressed as:

[0095] C = c + R·(t + L - T) (2)

[0096] Where, R is the unit profit of the mechanical equipment.

[0097] In the concept of reinforcement learning, the current state is a summary of all previous states, that is, the next state depends only on the current state. Therefore, it is impossible to set a time sliding window to improve the prediction stability like ordinary time series prediction. However, in order to reduce the impact of special circumstances on system maintenance and improve the robustness of the system, the present invention designs a state sliding window (SST) to replace the time sliding window. In general, the state of the entire interval [TL-SST, T-L+SST] represents the state at the TL time point. When the service time t of the real-time maintenance framework reaches this interval, it is considered that the framework predicts RUL=L.

[0098] This real-time ordering framework sets a negative reward function. For example, when RUL=L, the framework can choose two actions: continue to run or order with warning. When the action is to continue to run, the mechanical equipment continues to work and no cost is incurred. When the action is to order with warning, the mechanical equipment sends an order request with a total cost of c, and a negative reward function r=-c is set. In order to improve the training effect and correctly guide the actions of the intelligent agent, the present invention slightly modifies the cost function and converts it into a reward function. In the above case, it is the most ideal situation to choose to order with warning when the framework predicts RUL=L. Choosing to order with warning in other states is a suboptimal strategy. Therefore, the reward function of the real-time ordering framework can be defined as follows:

[0099]

[0100] Among them, r t is the evaluation value, t is the service life of the sample mechanical equipment, a t is the sample predictive maintenance result of the sample mechanical equipment, C is the total ordering cost of the sample mechanical equipment, T is the life of the mechanical equipment, L is the advance ordering period, SST is the status window, Order means warning ordering, and Continue means continued operation.

[0101] Among them, the process of real-time ordering framework is as follows:

[0102]

[0103]

[0104] Input of the real-time ordering framework: belief state ζ = [x1, x2, …, x LD ] T , a t-1 (Action selected in the previous state) c, I, L, R.

[0105] Output of the real-time ordering framework: trained DT-PPO, updated policy network parameters and value network parameters, ordering cost C and ordering point T0*=t. It should be noted that the ordering point is the moment when the predicted order is issued, and the policy network is the real-time ordering framework.

[0106] Specifically, the real-time ordering framework performs a total of max_epoch training, and each training process processes LD data sequences from t = 1 to t = LD. t =Order, at which point T*=t represents the order point.

[0107] According to the random strategy function π θ (a t |s t ) Formula 4 selects action a, where π θ It is a policy model determined by the parameter θ, usually a neural network. The random strategy adopted by the Proximal Policy Optimization (PPO) algorithm is that the agent selects action a according to the probability distribution of the current state s.

[0108] π θ (a t |s t )=P(a t =a|s t =s) (4)

[0109] The present invention designs an s-gpolicy that combines deterministic and random strategies. The random strategy is used in the training phase to achieve the purpose of exploration, and the deterministic strategy is used in the test phase to achieve the purpose of utilization. The s-gpolicy can be expressed as follows:

[0110]

[0111] Among them, softmax is a random strategy, and greedy is a deterministic strategy. According to s-gpolicy, select action a t , execute action a t .

[0112] The advantage function A can be calculated according to formula (6): t GAE(λ) .

[0113]

[0114]

[0115] Where r is the discount factor, λ is the GAE parameter, is the V - value part of the error at time step t + l. The calculation of the V - value part will use the value function V(s).

[0116] Calculate the ratio according to the clip method in formula (8), and the clipped ratio is used for the calculation of the policy function loss and the value function loss.

[0117]

[0118] where, π θ (a t |s t ) represents the probability of selecting action a t in state s t according to the new policy parameter θ. π θold (a t |s t ) is the probability of selecting action a old in state s t under the old policy function θ t . r t (θ) represents the unclipped version of the ratio calculated according to the new policy parameter θ multiplied by the advantage function. The role of the Clip function is to limit the ratio within the interval [1 - ε, 1+ε] to ensure that the policy update is not too drastic.

[0119] In an exemplary embodiment, the construction process of the real - time maintenance framework is as follows:

[0120] If the predictive maintenance result of the real - time maintenance framework is to select shutdown maintenance when t < T, there is a replacement cost at this time, and the total cost can be expressed as:

[0121] C = cr (9)

[0122] Regardless of whether the predictive maintenance result of the real - time maintenance framework is to select continuous operation or shutdown maintenance when t = T, there are replacement costs and reset costs at this time, and the total cost can be expressed as:

[0123] C = cr + cf (10)

[0124] The real-time maintenance framework sets a negative reward function. For example, when t=T, the real-time maintenance framework can choose two actions: continue to run or stop for maintenance. When the action is to continue to run, the mechanical equipment continues to work and no cost is incurred. When the action is to stop for maintenance, the mechanical equipment stops for maintenance, and the total cost is cr+cf. A negative reward function r=-(cr+cf) is set. In order to improve the training effect and correctly guide the actions of the intelligent agent, the present invention slightly modifies the cost function and converts it into a reward function. In the above case, it is the most ideal situation for the framework to choose to stop for maintenance at t=T-1. At this time, the ordered spare parts have just arrived. Choosing to stop for maintenance in other states is a suboptimal strategy. The real-time maintenance framework also replaces the time sliding window with the state sliding window. The reward function of the real-time maintenance framework can be defined as follows:

[0125]

[0126] Among them, r t is the evaluation value, t is the service life of the sample mechanical equipment, a t is the sample predictive maintenance result of the sample mechanical equipment, cr is the replacement cost of the sample mechanical equipment, cf is the replacement cost, SST is the status window, Replace means warning maintenance, and Continue means continued operation.

[0127] The process of real-time maintenance framework is as follows:

[0128]

[0129]

[0130] Input of the real-time maintenance framework: belief state ζ = [x1, x2, ..., x LD ] T , α t-1 (action selected in the previous state), cr, cf.

[0131] Output of the real-time maintenance framework: trained DT-PPO, updated policy network parameters and value network parameters, maintenance point T0*=t; it should be noted that the maintenance point is the moment when the early warning maintenance is issued, and the policy network in this embodiment is a real-time maintenance framework.

[0132] Specifically, the real-time maintenance framework performs a total of max_epoch training, and each training process processes LD data sequences from t = 1 to t = LD. t = "Replace", at which point T* = t represents the maintenance point.

[0133] According to formula (4), the strategy function π θ (a t |st ) select action a, where π θ It is a policy model determined by the parameter θ, usually a neural network. The random strategy adopted by the PPO algorithm is that the agent selects action a according to the probability distribution of the current state s.

[0134] The advantage function is calculated according to formula (6): The ratio calculated by the clip method in formula (8) and the clipped ratio are used to calculate the policy function loss; then the policy function loss and value function loss are calculated according to the min method in formula 8.

[0135] The current state of the sample and the predictive maintenance results are input into the value network, and the evaluation value is determined through the reward function in the value network; the strategy loss is determined according to the sample predictive maintenance results and the actual maintenance results of the sample, and the value loss is determined according to the evaluation value; the parameters of the initial proximal policy network are updated according to the strategy loss, and the parameters of the value network are updated according to the value loss; the updated initial proximal policy network model is determined as the real-time conversion-proximal policy optimization network model.

[0136] In an exemplary embodiment, the sample predictive maintenance result is the action output by the policy function, and the reward function in the value network is the r used in the ordering strategy and maintenance strategy above. t , according to the comparison between the actions output in the policy function and the actions output in the s-gpolicy above, the policy loss and value loss are determined; it should be noted that PPO is the real-time ordering framework and the real-time maintenance framework.

[0137] In an exemplary embodiment, the embodiment includes: obtaining a test belief state of a test mechanical equipment; inputting the test belief state into a real-time conversion-proximal strategy optimization network model to obtain action data of the test mechanical equipment; the action data is a determined value of each behavior of the test mechanical equipment; the behavior includes warning ordering, shutdown maintenance and continued operation; determining the predictive maintenance results of the test samples through a random strategy; determining the actual maintenance results of the test samples through a deterministic strategy; and determining the performance evaluation results of the real-time conversion-proximal strategy optimization network model based on the predictive maintenance results of the test samples and the actual maintenance results of the test samples.

[0138] Specifically, the prediction accuracy of the real-time conversion-proximal policy optimization network model can be determined according to the predictive maintenance results of the test samples and the actual maintenance results of the test samples. If the accuracy is greater than the preset accuracy threshold, the performance evaluation result of the real-time conversion-proximal policy optimization network model is determined to be excellent. If the prediction accuracy of the real-time conversion-proximal policy optimization network model is less than or equal to the preset accuracy threshold, the performance evaluation result of the real-time conversion-proximal policy optimization network model is determined to be poor, that is, the real-time conversion-proximal policy optimization network model needs to be retrained.

[0139] In an exemplary embodiment, the present invention also provides a flowchart of a real-time sensor-driven predictive maintenance method, such as Figure 3 As shown, this embodiment specifically includes: collecting the initial states of multiple sensors of mechanical equipment under different operating states, using a state transition network to convert the initial states into corresponding belief states, inputting training samples into DT-PPO for quality assessment of generated samples, and using test samples to output predictive maintenance decisions for test samples.

[0140] In order to verify the effectiveness of the predictive maintenance method for mechanical equipment provided by the present invention, the present invention uses a data set (Commercial Modular Aero-Propulsion System Simulation, C-MAPSS) to carry out method verification. C-MAPSS is derived from the transonic turbofan engine (Turbofan engine). The Turbofan engine is a modern gasoline turbine engine used by the National Aeronautics and Space Administration (NASA). The structure diagram of the simulated aircraft engine is as follows: Figure 4 As shown, the simulated aircraft engine includes fan 1, combustion chamber 2, low-pressure rotor speed 3, low-pressure turbine 4, low-pressure compressor 5, high-pressure compressor 6, high-pressure rotor speed 7, high-pressure turbine 8, and nozzle 9. The data set includes the time series of each engine. All engines are of the same type, but during the manufacturing process, the initial wear and tear and differences of each engine are different, which the user does not know. There are three optional configurations that can be used to change the performance of each engine. Each engine has 21 sensors, and when the engine is running, the sensors collect measurement data related to the engine status. There is some sensor noise in the collected data. Gradually, each engine will have some deficiencies, which can be found from the sensor readings. The time series ends at a certain time before the failure occurs. Table 1 shows the simulated aircraft engine data set, including the introduction of the C-MAPSS data set.

[0141] In actual monitoring, since the output of some sensors is fixed, they cannot provide effective information for the framework. In order to reduce the computational complexity of the framework, this paper selects 14 sensor parameters to construct samples and extracts degradation feature information through the proposed framework MCIBIGRUN. The corresponding numbers of sensors are 2, 3, 4, 7, 8, 9, 11, 12, 13, 14, 15, 17, 20 and 21 respectively.

[0142] Table 1

[0143]

[0144] Then, a state transition network model was established, and the training sample set was used for network model training. As a deep autoencoder, the state transition network selectively extracts multi-sensory features of different scales and outputs belief states. The belief state of TransStateNet is the same size as the initial state and serves as the input of the state space of DT-PPO. The Adam optimizer was used to perform stochastic gradient optimization on the network, and the initial learning rate was 0.01 and the weight decay rate was 0.01. The training was iterated 1000 times, and gradient penalty regularization was used to improve training stability.

[0145] In order to prove the effectiveness of the proposed real-time ordering framework, the present invention first defines a set of performance indicators for comparison. The first is the root mean square error (RMSE), the second is the scoring function (Score), and the third is the mean deviation (MBE). The experimental results of the real-time ordering framework are related to the ordering price of mechanical equipment, the capital utilization rate of the market, and the profit generated per unit service life. The price of turbofan engines varies greatly due to factors such as its type, performance, manufacturer, newness, and whether it includes after-sales support. The capital utilization rate is the interest rate of the central bank, which is relatively stable. The unit profit is affected by many factors, including customer income, operating costs, and market strategies. In order to make a better comparison, the present invention considers different cost combinations c = {100,500}, I = 0.05, R = {0.5,1}.

[0146] The experimental results are shown in Table 2. Table 2 shows the experimental results of the real-time ordering framework, which shows the RMSE, Score, and MBE of the real-time ordering framework under different cost combinations. When c = 100, the increase in R may help reduce the prediction error. When c = 500, the increase in R may lead to an increase in the prediction error. All three indicators show such a trend; when R = 0.5, the increase in c may help reduce the prediction error. When R = 1, the increase in c may lead to an increase in the prediction error. The three indicators also show such a trend. This may mean that under the condition that one parameter is constant, the change of another parameter does not have a monotonic effect on the performance indicator, and further analysis may be needed to determine its impact pattern. In addition, under all cost combinations, MBE is greater than 0, indicating that the framework always tends to issue order warnings in advance.

[0147] Table 2

[0148] c I R L PM MSE 100 0.05 0.5 20 191.40 100 0.05 1 20 147.20 500 0.05 0.5 20 136.40 500 0.05 1 20 240.40 MAPE 100 0.05 0.5 20 20％ 100 0.05 1 20 20％ 500 0.05 0.5 20 20％ 500 0.05 1 20 20％ MBE 100 0.05 0.5 20 5.40 100 0.05 1 20 2.80 500 0.05 0.5 20 2.80 500 0.05 1 20 7.20

[0149] In order to prove the effectiveness of the proposed control decision framework, the present invention first defines a set of performance indicators for comparison. The first is the average maintenance cost rate (AMCR), the second is the average wasted life (AWOL), the third is the average planned replacement number (ASRT), and the fourth is the average unplanned replacement number (AUSRT). In order to prove the effectiveness of the proposed control decision, the present invention first defines a set of benchmark strategies for comparison. The first is the ideal maintenance strategy (IMC), the second is the corrective maintenance strategy (CMC), and the third is the planned maintenance strategy (SM).

[0150] The experimental results are shown in Table 3. Table 3 is the experimental results of the real-time ordering framework, which shows the AC, AWOL, ASRT, and AUSRT of the framework under different cost combinations. The first result is the average maintenance cost rate for five different maintenance strategies and four different cost combinations. As expected, the ideal replacement policy IMC provides the lowest cost. The present invention provides the second best result and performs better than the planned maintenance strategy and the corrective maintenance strategy. In addition, the average wasted life of the four benchmark strategies is also reported. As shown in the table, the corrective maintenance strategy will not waste life. The ideal maintenance cost strategy IMC provides the second best result. The predictive maintenance strategy of the present invention ranks behind the two and is better than the two cases of the planned maintenance strategy. Finally, the framework of the present invention performs well in terms of the average number of planned replacements and the average number of unplanned replacements.

[0151] Table 3

[0152]

[0153] The content of the present invention is mainly divided into two parts. The first part is a state transition network, which is used to extract multi-dimensional features of sensors, eliminate redundant degradation information and convert the initial state into a belief state; the second part is to establish an improved state transition-proximal strategy optimization network model, including a real-time ordering framework and a real-time maintenance framework, and integrate the belief state of mechanical equipment into the real-time ordering and real-time maintenance framework. The real-time ordering framework is used to issue order requests; the real-time maintenance framework is used to decide when the equipment should be shut down for maintenance.

[0154] Among them, the convolutional neural network in the state transition network model is the CNN module, the residual neural network in the state transition network model is the ResNet module, the self-attention mechanism module in the state transition network model is the Self-Attention module, and the bidirectional gated recurrent unit in the state transition network model is the BiGRU module.

[0155] Reference Figure 5 As shown in the figure, PPO adopts the actor-critic architecture. The present invention uses two single hidden layer neural network structures to test our proposed method to find the best network that represents the correct mapping between state and action. In addition to the state processed by the state transition network, the input layer also contains the action selected by PPO based on the previous state. The two neural networks have the same number of input and hidden layer neurons. The policy network outputs the action probability distribution, that is, the probability of executing each action in a given state. This network is usually the output of the softmax function. The value function network outputs the estimated state value function, that is, the expected value of the total reward obtained in the future under the current state. This is a scalar output that reflects how much reward the agent is expected to accumulate in the current state. Taking into account the limitations of each architecture, the number of hidden neurons and hidden layers of the Actor-Critic architecture is determined by cross-validation to better represent the interaction between the input data and its target value.

[0156] The beneficial effects of the predictive maintenance method driven by real-time sensors of the present invention are as follows: the method relies on the real-time data sharing of sensors and the powerful feature extraction capabilities of deep learning and the dynamic decision-making capabilities of reinforcement learning to convert the real-time sensor data collected from the monitoring sensors into the basis of real-time maintenance and real-time decision-making. Through the state transition network module, the framework can effectively process data from multiple sensors, perform feature extraction and prediction of degradation features, and solve the high-dimensional and multi-scale problems that are difficult to handle in traditional methods. It can be seen from the experimental results that the framework shows good robustness under different cost combinations and fault modes, and can adapt to different operating conditions and external environments. The method can make full use of existing fault samples, flexibly mine sample features, and accurately match the local feature positions between images by embedding a multi-head attention mechanism, which significantly improves the feature extraction performance of the generative adversarial network and the local generation quality of the sample. Effectively improve the accuracy and stability of the deep learning fault diagnosis model in the case of extremely small samples. Although the embodiments of the present invention have been disclosed as above, they are not limited to the applications listed in the specification and the embodiments. They can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily implemented. Therefore, without departing from the general concept defined by the claims and the scope of equivalents, the present invention is not limited to the specific details and the illustrations shown and described herein.

[0157] In the predictive maintenance method for mechanical equipment provided by the present invention, a state transition network is constructed as a deep autoencoder to selectively extract multi-sensory features of sensors at different scales and output belief states; a real-time conversion-proximal policy optimization network model based on belief states is designed to establish a strategy for ordering spare parts or shutdown maintenance; an sg policy is designed for the real-time conversion-proximal policy optimization network model to guide action selection and use a state sliding window to process multi-sensor state sequence data, thereby reducing the uncertainty of the action space of the real-time conversion-proximal policy optimization network model, improving the robustness of the maintenance strategy, and avoiding accidents. This method solves the problem that the existing model for integrating data-driven remaining useful life into maintenance plans is too complex.

[0158] When applying the predictive maintenance method for mechanical equipment provided by the present invention, it is not necessary to Figure 1 The steps are executed in the order shown. The specific execution order of the steps can be determined according to needs, and the present invention does not limit this.

[0159] The above is a predictive maintenance method for mechanical equipment provided by one or more embodiments of the present invention. Based on the same idea, the present invention also provides a corresponding predictive maintenance device for mechanical equipment, such as Figure 6 shown.

[0160] Figure 6A schematic diagram of a predictive maintenance device for mechanical equipment provided by the present invention, the device 600 comprises:

[0161] An acquisition module 601 is used to acquire state data of mechanical equipment at multiple times;

[0162] Extraction module 602, used to input the state data into the state transition network model, extract multi-sensory features of different scales of the state data through the state data, and obtain the belief state; the belief state is a feature representation of the state data;

[0163] The prediction module 603 is used to input the belief state into the pre-trained real-time conversion-proximal strategy optimization network model to obtain the predictive maintenance results of the mechanical equipment.

[0164] The specific definition of a predictive maintenance device for a mechanical device can be found in the above definition of a predictive maintenance method for a mechanical device, which will not be repeated here. Each module in the above-mentioned predictive maintenance device for a mechanical device can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0165] The present invention also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 7 A predictive maintenance method for mechanical equipment is provided.

[0166] The present invention also provides Figure 7 The structural diagram of the computer device shown in FIG. Figure 7 As shown in the figure, at the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 A predictive maintenance method for mechanical equipment is provided.

[0167] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0168] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

Claims

1. A predictive maintenance method for mechanical equipment, characterized in that: include: Obtain status data of mechanical equipment at multiple times; The state data is input into a state transition network model, and multi-sensory features of different scales of the state data are extracted through the state transition network model to obtain a belief state; the belief state is a feature representation of the state data; The belief state is input into a pre-trained real-time conversion-proximal strategy optimization network model to obtain the predictive maintenance result of the mechanical equipment.

2. The method according to claim 1, characterized in that The obtaining of state data of the mechanical equipment at multiple times includes: Acquire original state data collected in real time by multiple sensors on the mechanical equipment at the multiple moments; The original state data is normalized to obtain state data of the mechanical equipment at multiple moments.

3. The method according to claim 1, characterized in that The state transition network model includes a convolutional neural network, a residual neural network, a self-attention mechanism module and a linear neural network module. The state transition network model is used to extract multi-sensory features of different scales of the state data to obtain a belief state, including: Inputting the state data into the convolutional neural network to obtain high-order features of the state data; Inputting the high-order features of the state data into the residual neural network to obtain the high-order optimized features of the state data; Inputting the high-order optimization features of the state data into the self-attention mechanism module to obtain the high-order belief state; The high-order belief state of the state data is input into the linear neural network to obtain the belief state.

4. The method according to claim 3, characterized in that The state transition network model also includes a bidirectional gated recurrent unit, and the method further includes: The belief state is input into the bidirectional gated cycle unit to obtain the remaining service life of the mechanical equipment.

5. The method according to claim 1, characterized in that: The real-time conversion-proximal strategy optimization network model includes a real-time ordering framework and a real-time maintenance framework; the belief state is input into the pre-trained real-time conversion-proximal strategy optimization network model to obtain the predictive maintenance result of the mechanical equipment, including: Inputting the belief state into the real-time ordering framework to obtain an ordering strategy for the mechanical equipment, and determining the ordering strategy as the predictive ordering result; The belief state is input into the real-time maintenance framework to obtain a maintenance strategy for the mechanical equipment, and the maintenance strategy is determined as the predictive maintenance result.

6. The method according to claim 1, characterized in that The construction process of the real-time conversion-proximal strategy optimization network model includes: Obtain sample belief states of sample mechanical devices; Inputting the sample belief state into the policy network to obtain a sample predictive maintenance result of the sample mechanical equipment; Inputting the current state and the sample predictive maintenance result into the value network, and determining the evaluation value through the reward function in the value network; Determine the strategy loss according to the sample predictive maintenance result and the sample real maintenance result, and determine the value loss according to the evaluation value; updating parameters of the policy network according to the policy loss, and updating parameters of the value network according to the value loss; The updated policy network and value network are determined as the real-time conversion-proximal policy optimization network model.

7. The method according to claim 6, characterized in that The step of inputting the sample belief state into the policy network to obtain the sample predictive maintenance result of the sample mechanical equipment includes: Input the sample belief state into the policy network to obtain the action data of the sample mechanical equipment; the action data is the probability value of each behavior of the sample mechanical equipment; the behavior includes early warning ordering, shutdown maintenance and continued operation; According to the probability value of each behavior, the sample predictive maintenance result is determined through a random strategy.

8. The method according to claim 6, characterized in that The method further comprises: Get the test belief status of the test mechanical equipment; Inputting the test belief state into the real-time conversion-proximal strategy optimization network model to obtain the action data of the test mechanical equipment; the action data is a determined value of each behavior of the test mechanical equipment; the behavior includes continued use, maintenance and ordering; Determine the predictive maintenance result of the test sample by a random strategy; Determining the actual maintenance result of the test sample by a deterministic strategy; According to the test sample predictive maintenance result and the test sample actual maintenance result, the performance evaluation result of the real-time conversion-proximal strategy optimization network model is determined.

9. The method according to claim 6, characterized in that The real-time conversion-proximal strategy optimization network model includes a real-time ordering framework, and the reward function includes: Among them, r t is the evaluation value, t is the service life of the sample mechanical equipment, a t is the sample predictive maintenance result of the sample mechanical equipment, C is the total ordering cost of the sample mechanical equipment, T is the life of the mechanical equipment, L is the advance ordering period, SST is the status window, Order means warning ordering, and Continue means continued operation.

10. The method according to claim 6, characterized in that The real-time conversion-proximal strategy optimization network model includes a real-time maintenance framework, and the reward function includes: Among them, r t is the evaluation value, t is the service life of the sample mechanical equipment, a t is the sample predictive maintenance result of the sample mechanical equipment, cr is the replacement cost of the sample mechanical equipment, cf is the replacement cost, SST is the status window, Replace indicates early warning maintenance, and Continue indicates continued operation.

Images