Complex equipment data processing method based on CITIN

Abstract

The invention relates to the technical field of data processing of complex equipment, in particular to a method for filling data of complex industrial equipment by applying a constrained time interval perception time sequence interpolation network CITIN to data filling of the complex industrial equipment, so that the possibility that a neural network generates unreasonable values in the filling process in the subsequent data processing process is reduced, and the data filling efficiency is improved. According to the complex equipment data processing method based on the CITIN, the data processing accuracy and stability of industrial equipment are improved, the influence of different running stages and different durations of an engine in each flight on performance degradation of the engine is considered, and the missing part in engine performance data is more accurately filled; meanwhile, aiming at the problem that unreasonable values occur sometimes due to the uninterpretability of the neural network, the PCHIP is used for restraining the range of filling results generated by the neural network in the training stage of the filling network, and the possibility that the unreasonable values are generated by ITIN is reduced.

Description

Technical field:

[0001] The present invention relates to the technical field of complex equipment data processing, and specifically to a complex equipment data processing method based on CITIN, which applies a constrained interval-aware temporal imputation network (CITIN) to complex industrial equipment data filling, reduces the possibility of a neural network generating unreasonable values ​​in the filling process during subsequent data processing, and improves the accuracy and stability of industrial equipment data processing. Background technology:

[0002] The accurate collection of environmental conditions and machine status by sensors promotes the implementation of condition monitoring and health management for complex equipment. By adding a large number of sensors during the design process, researchers can monitor the status of complex equipment during use, thereby providing a basis for subsequent maintenance and support work. However, due to possible failures during sensor operation, negligence during the manual operation stage, and regulations on information sharing among multiple departments, the status data obtained may be missing. This problem of incomplete data brings difficulties to subsequent analysis work.

[0003] There are many ways to handle data with missing values. Directly deleting some of the data with missing values ​​is the simplest and easiest to operate, but this is not applicable in all scenarios, especially time series data such as aircraft engine operation data. Because the data characteristics of multivariate time series are manifested as continuity in time and correlation between different variables, if some data are deleted directly, this continuous relationship may be interrupted, resulting in large deviations in subsequent work. For aircraft engine data, the lack of all data for certain flights may make subsequent anomaly detection, fault diagnosis, and life prediction difficult to carry out. Therefore, filling in data containing missing values ​​is regarded as a better way to deal with incomplete performance data of aircraft engines.

[0004] The use of statistical methods to fill missing values ​​is the most common in industry because it has good interpretability and is easy to calculate. The most basic statistical filling method is to use descriptive statistics to fill the missing parts, such as forward filling, backward filling, mean filling, mode filling, etc. This type of filling method runs fast and can meet the requirement that the final data has no missing values. However, these methods do not consider the previous and next correlations in the time dimension and feature dimension, and the filling effect is not ideal in most cases.

[0005] In the prior art, methods based on recurrent neural networks (RNN) are widely used for filling in. This method first uses simple statistical methods (such as mean filling, forward filling, cubic spline interpolation, etc.) to preliminarily fill in missing values, and then uses the filled complete data to train the RNN. There are two main problems with the current method applied to aircraft engines. First, the current neural network is unstable in actual industrial data, and sometimes the filled value exceeds the normal value range, that is, unreasonable filled values ​​are generated. Although the unreasonable values ​​can be uniformly processed after the filling is completed, this processing method does not solve the problem of filling, but re-fills a feature value to solve the unreasonable value. Secondly, the use time of aircraft engines is not exactly the same in each flight, and multiple working conditions of different durations accumulate in each flight. For civil aircraft, a complete flight process mainly includes takeoff, climb, cruise, descent and landing (takeoff, climb, cruise, descent, landing), and in some special cases, it also includes go-around, circling and holding (go-around, holding), ground test, etc. In these states, various differences lead to inconsistencies in the actual running time and performance degradation of two adjacent flight data. However, the traditional filling method does not handle the same situation when processing equally spaced sampling data. Summary of the invention:

[0006] In view of the shortcomings and deficiencies in the prior art, the present invention proposes a complex equipment data processing method based on CITIN which can effectively reduce the possibility of unreasonable values.

[0007] The present invention is achieved by the following measures:

[0008] A complex equipment data processing method based on CITIN, characterized by comprising the following steps:

[0009] Step 1: Get the original data;

[0010] Step 2: Data preprocessing: split the data set to obtain the test set and training set, splice the original data sequence according to the training set, validation set and test set, use the training set data as the standard, standardize and normalize, weaken the data differences in the feature dimension, improve the filling effect of the neural network, and use the sliding window to cut the data of different lengths into the same length for easy input into the neural network;

[0011] Step 3: Establish a filling network model and train it. The filling network model introduces Transformer to process the time interval information based on GRU, and combines the result with the missing label matrix to adjust the hidden state of the missing part and the input weight to form a time interval-aware temporal interpolation network. Secondly, the piecewise cubic Hermite interpolation polynomial PCHIP is introduced into the training process of the time interval-aware temporal interpolation network to constrain the range of filling result generation;

[0012] During training, the incomplete training set is first input into two time interval-aware temporal interpolation networks to obtain the reconstructed data and temporary filling data output by the network, and then the MAE of the reconstructed data and the original data is used as part of the loss function, and the MAE of the consistency of the filling results of the two networks is used as part of the loss function. The two sets of filling results are averaged and used as the temporary filling results in the training stage. Subsequently, the incomplete data is input into the PCHIP unit to form temporary filling data, and the MSE of the temporary filling results and the filling results of the PCHIP unit are used as part of the loss function to jointly optimize the network parameters of the two time interval-aware temporal interpolation networks; Step 4: Application of the filling network: First, the test data is input into the two time interval-aware temporal interpolation networks that have been trained to obtain the reconstructed data and the temporary filling data, and then the temporary filling results in the two networks are averaged to be used as the final filling result of the test set.

[0013] The input of the filling network model of the present invention includes incomplete data X, missing label matrix M, time interval matrix Δ and absolute time s, multivariate time series is incomplete data, where T is the length of the data in the time dimension, D is the feature dimension of the data, and the missing marker matrix M is the position where the data of the original data X is missing. The shape is the same as X. For the data of the dth dimension at time t, have

[0014]

[0015] Δ is represented as the time interval matrix of X, which has the same shape as X. The time interval matrix Δ is calculated for each dimension separately, and the calculation rules for the corresponding position include three rules: first, the time interval δ1 of the first timestamp is set to 0; second, for the subsequent timestamp t, if there is data in the sample sequence X of the dimension corresponding to the previous timestamp of the timestamp, the current absolute time is subtracted from the absolute time of t-1 as the value corresponding to Δ; finally, if the data of X at the corresponding position of t-1 is empty, on the basis of subtracting the current absolute time from the absolute time of t-1, the time interval value of the previous timestamp is also added. The calculation of Δ in the dth dimension at time t is expressed as

[0016] Among them, s t represents the absolute time at the tth moment; Represents the time interval matrix of the dth dimension at time t.

[0017] The input data of the network model filled in the present invention is marked as The positions of these data are marked using the indicator matrix I. The shape of the indicator matrix is ​​the same as the original data. The data positions where the observed values ​​exist but are hidden in actual use for prediction are represented as 1, and the remaining positions are represented as 0. The indicator matrix I at time t is represented by the dth dimension:

[0018]

[0019] The filling network model of the present invention includes two time interval-aware time series interpolation networks, one of the two networks is used to capture the law from the forward direction of the training data, and the other is used to capture the law from the reverse direction. The two networks each provide a temporary filling result, and a PCHIP unit is also provided for calculating the filling result using the statistical method. The inputs of the two time interval-aware time series interpolation networks include the original data X, the time interval matrix Δ and the missing mark matrix M. The first time interval-aware time series interpolation network INIT1 starts from the first timestamp and processes all the features of each timestamp in sequence until the last one, representing the process of forwardly capturing the data relationship in the time dimension, and the hidden layer H of the INIT structure is processed. t Input to the next layer, and convert the output c1~c T Temporary imputed data1 is spliced ​​together according to the time dimension. Expressed as

[0020] C=Concat(c1,c2,…,c T ),

[0021]

[0022] Where C represents the matrix after the output of each step of the GRU model is concatenated; Concat(·) represents concatenating vectors into matrices, ⊙ represents multiplication of corresponding positions of matrices of the same size, and the second time interval-aware temporal interpolation network INIT2 has the opposite processing process to the first one. It learns the time series from back to front in the time dimension to capture the impact of future data on the filling results of past data. The filling result of the second time interval-aware temporal interpolation network INIT2 is expressed as The input of the PCHIP unit processing flow includes the original data X and the absolute time vector s corresponding to each timestamp. The PCHIP unit needs to process each dimension of X separately. When processing the data of the dth dimension, The derivative is calculated for each point of , and then each coefficient of the cubic function is calculated based on the derivative and the slope, and finally the position s containing the missing data is t Substitute the Hermite polynomial composed of multiple functions and solve the corresponding position After all missing values ​​are filled, the filling results can be obtained

[0023] The method for processing all feature dimension data at each moment by the time interval-aware time series interpolation network in the present invention includes, in addition to the original input of each timestamp, the information transmitted by the historical timestamp, where H t-1 Represents the hidden features transmitted by the previous timestamp. When processing the first timestamp, H transmitted by the historical timestamp t-1 is set to 0. The following is the specific process of processing each timestamp data: Step ①: The hidden variables of the previous layer are converted to x after a linear layer dimension reduction. h , x h The size of is the same as the input x;

[0024] Step ②: Use the existing value in x to replace the signal attenuation of the features of the hidden layer of the previous layer x h Reconstruct the data at the corresponding position to form the input x c Then, after a linear layer transformation, the output z of the network is obtained h , expressed as

[0025] x c =x⊙m+x h ⊙(1-m)

[0026] z h =Linear(x c )

[0027] Step ③: Generate weight γ according to the current time interval δ x , used to balance the input z h and historical information x h The important weights of x The concatenated missing marker matrix m is transformed to the same matrix as x through a linear transformation and a Sigmoid (·) activation function. h and z h The same size, denoted as β, is used to merge the features of Step ① and Step ②. The merging formula is

[0028] c h =β·x h +(1-β)·z h ,

[0029] Step ④: Change the c in Step ③h After observation value replacement, it is expressed as c c , c c Concatenate with the missing token m in the feature dimension as the input X of the GRU i , and the H of the previous layer t-1 Input into GRU to get H of the current timestamp t .

[0030] In Step ③ of the present invention, the overall input is δ, which is the multi-sensor data time interval vector for each timestamp; the output is the time attenuation factor γ x , the self-attention mechanism is introduced to enhance the network input's perception of time intervals and improve the ability to weigh the hidden variables and input weights of GRU. The left side of the self-attention mechanism-enhanced network is a basic linear layer used to generate a temporary γ, which is calculated as follows:

[0031] γ=ReLU(δW d ⊙I+b d )

[0032] Where W d and b d are the weights and biases of the network used to provide the attenuation factor; I is the unit matrix, which is used to maintain the existence of the time interval information; ReLU (@) is the activation function. The right side of the self-attention mechanism enhanced network is the self-attention mechanism. First, the time interval δ is input into three linear layers in sequence, and the outputs of the three linear layers are used as Query, Key and Value respectively. Then Q and K are dot-producted and divided by the scaling factor To avoid the dot product result being too large, it is then normalized by Softmax(·). The result obtained is the weight parameter of the self-attention mechanism. The weight parameter is dot-producted with V and converted to the same size as the output of the basic linear layer through a linear transformation. Finally, the results on the left and right sides are added together, and the distribution range of the results is adjusted through an exponential function to form the final γ x .

[0033] In Step ③ of the present invention, in GRU, the hidden state H t-1 and input X i Going a step further, the GRU contains a reset gate R t and an update gate Z t , the purpose of the reset gate is to calculate the candidate hidden state The purpose of the update gate is to act as a weight to adjust the newly generated H t Middle, H t-1 and The weights of both structures are two fully connected layers using the Sigmoid (·) activation function, expressed as

[0034] R t =Sigmoid(X i W xr +H t-1 W hr +b r )

[0035] Z t =Sigmoid(X i W xz +H t-1 W hz +b z )

[0036] Where W xr , W hr and b r is the network parameter of the reset gate, W xz , W hz and b z is the network parameter of the update gate, integrates the reset gate with the hidden state, and calculates the candidate hidden state The formula is

[0037]

[0038] Where W xh , W hh and b h is the network parameter for calculating the candidate hidden state; ⊙ is the element-wise product operator, and the calculation method for the GRU output hidden state is

[0039]

[0040] The PCHIP unit described in the present invention is used for smooth interpolation processing of data, specifically: constructing a cubic polynomial in the interval between every two observation time stamps, and making these polynomials have the same numerical value and derivative on the observation value, including four steps: splitting the original input to provide incomplete sequence data; calculating the derivative of each existing point in the data, ensuring that the derivatives of the data point on the left and right sides are consistent; constructing a cubic Hermite polynomial in each interval, and ensuring that the same function value and derivative value are present at the position corresponding to each numerical value; using the cubic polynomial to calculate the interpolation result at any position.

[0041] The present invention proposes a constrained time interval-aware timing interpolation network for complex equipment such as aircraft engine performance data. The network not only takes into account the impact of engine performance degradation at different stages and durations of operation of each flight, but also more accurately fills in the missing parts of the engine performance data. At the same time, in view of the problem that unreasonable values ​​sometimes appear in neural networks due to their unexplainability, PCHIP is used in the training stage of the filling network to constrain the range of filling results generated by the neural network, thereby reducing the possibility of ITIN generating unreasonable values. Description of the drawings:

[0042] Figure 1 It is a schematic diagram of the model in the present invention.

[0043] Figure 2 It is an internal flow chart of the Interval-aware TemporalImputation Network in the present invention.

[0044] Figure 3 It is a schematic diagram of the self-attention mechanism enhanced network in the present invention.

[0045] Figure 4 is a flow chart of an embodiment of the present invention.

[0046] Figure 5 It is a basic structural schematic diagram of a turbofan engine in an embodiment of the present invention.

[0047] Figure 6 10 experimental results of different methods in the embodiments of the present invention on a civil aviation engine data set, (a) is a schematic diagram of the MSE indicator, and (b) is a schematic diagram of the MAE indicator.

[0048] Figure 7 3 is a comparison chart of the experimental results of CITIN and BRITS under different signal-to-noise ratios in the embodiment of the present invention.

[0049] Figure 8 These are the results of ten experiments on a civil aviation engine data set using different methods in an embodiment of the present invention, where (a) is a schematic diagram of the MSE indicator and (b) is a schematic diagram of the MAE indicator.

[0050] Fig. 9 1 is a histogram and standard deviation of MSE and MAE at different signal-to-noise ratios on a real civil aviation data set in an embodiment of the present invention. Specific implementation method:

[0051] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0052] The present invention includes a constrained time interval-aware temporal interpolation network unit, a PCHIP (Piecewise Cubic Hermite Interpolating Polynomial) unit and a redesigned loss function, wherein the network input mainly includes incomplete data X, a missing label matrix M, a time interval matrix Δ and an absolute time s. Multivariate time series is incomplete data, where T is the length of the data in the time dimension and D is the feature dimension of the data. The missing marker matrix M is used to mark the location where the data of the original data X is missing, and its shape is the same as X. For the data of the dth dimension at time t, have

[0053]

[0054] Δ represents the time interval matrix of X, and its shape is the same as X. Assume that there is a sample sequence X as shown on the left side of Table 1, x1~x6 are the split vectors of matrix X in the time dimension, " / " represents missing data, otherwise it represents the existence of data corresponding to the position, the absolute time s=[0,3,7,12,18,20] of x1~x6, and the corresponding time interval Δ is shown on the right side of Table 1. The time interval matrix Δ is calculated separately for each dimension, and the calculation rules for the corresponding position include three rules: first, the time interval δ1 of the first timestamp is set to 0; second, for the subsequent timestamp t, if there is data in the sample sequence X of the dimension corresponding to the previous timestamp of the timestamp, the current absolute time is subtracted from the absolute time of t-1 as the value corresponding to Δ; finally, if the data of X at the corresponding position of t-1 is empty, on the basis of subtracting the current absolute time from the absolute time of t-1, the time interval value of the previous timestamp must be added. The calculation of Δ in the dth dimension at time t can be expressed as

[0055]

[0056] Among them, s t represents the absolute time at the tth moment; Represents the time interval matrix of the dth dimension at time t.

[0057] Table 1 An example of constructing time intervals based on missing data locations and absolute times

[0058]

[0059] In addition to M and Δ derived from X, a portion of observations in the validation and test sets need to be set as missing to verify and test the ability of the neural network for filling in the gaps and to evaluate the filling effects of different methods. The manually masked data are marked as The positions of these data are marked using the indicator matrix I. The shape of the indicator matrix is ​​the same as the original data. The data positions where the observed values ​​exist but are hidden in actual use for prediction are represented as 1, and the rest are represented as 0. The indicator matrix I at time t is represented by the dth dimension

[0060] Attached Figure 1 The main components of the proposed method model are presented, and the use of the input data introduced above is demonstrated. Figure 1 In the upper part, the middle is the input of each module, including the four types introduced above; the upper and lower parts are each a time interval-aware time series interpolation network, one for capturing the rules from the forward direction of the training data, and the other for capturing the rules from the reverse direction, and each of the two networks provides a temporary filling result; the right side is the PCHIP unit, which is used to calculate the filling results using statistical methods. Figure 1 The rectangular block in the middle of the upper half is the input of the two Interval-aware Temporal Imputation Networks, including the original data X, the time interval matrix Δ, and the missing mark matrix M. The figure has been split into x for each timestamp in the time dimension. t ,δ t ,m t Interval-aware Temporal ImputationNetwork1 (INIT1) starts from the first timestamp and processes all the features of each timestamp in sequence until the last one, which represents the process of positively capturing data relationships in the time dimension. t Input to the next layer, and convert the output c1~c T According to the time dimension, Temporary imputed data1 (i.e. ) can be expressed as

[0061] C=Concat(c1,c2,…,c T )

[0062]

[0063] Where C represents the matrix after the output of each step of the GRU model is concatenated; Concat(·) represents concatenating vectors into matrices, and ⊙ represents multiplying corresponding positions of matrices of the same size. Interval-aware Temporal ImputationNetwork2 (INIT2) is the opposite of the processing process of the first Cell. It learns the time series from back to front in the time dimension to capture the impact of future data on the filling results of past data. The filling result of Interval-aware Temporal ImputationNetwork2 (INIT2) is expressed as

[0064] The input of the PCHIP unit processing flow in the present invention includes the original data X and the absolute time vector s corresponding to each timestamp. The PCHIP unit needs to process each dimension of X separately. When processing the data of the dth dimension, The derivative is calculated for each point of , and then each coefficient of the cubic function is calculated based on the derivative and the slope, and finally the position s containing the missing data is t Substitute the Hermite polynomial composed of multiple functions and solve the corresponding position After all missing values ​​are filled, the filling results can be obtained

[0065] Figure 1 The lower part is the training and verification process of the upper part module. The error metrics of the two sets of temporary filling results and reconstruction results obtained in the upper part, as well as the error metrics of the filling results in the upper part and PCHIP are used together to form a loss function to train two Interval-aware Temporal Imputation Networks. After the training of the two networks is completed, the filling results of the two networks can be averaged as the final result, which is the Imputed Data marked with dotted lines in the figure.

[0066] Interval-aware Temporal Imputation Network is a network based on GRU. The typical feature of GRU is that it processes each timestamp step by step. In essence, it assumes that there is a regression relationship between the data of the current timestamp and the data of the processed timestamp. At time t, the original input contains x t , δ t , m t , representing the vector at time t in the original data, time interval, and missing marker matrix, respectively.

[0067] Figure 2This shows how to process all feature dimension data at each moment. In addition to the original input of each timestamp, it also includes the information transmitted by the historical timestamps. t-1 Represents the hidden features transmitted by the previous timestamp. When processing the first timestamp, H transmitted by the historical timestamp t-1 is set to 0. The following is the specific process for processing each timestamp data:

[0068] Step ①: The hidden variables of the previous layer are converted to x after dimension reduction through a linear layer h , x h The dimensions of are the same as the input x.

[0069] Step ②: Use the existing value in x to replace the signal attenuation of the features of the hidden layer of the previous layer x h Reconstruct the data at the corresponding position to form the input x c Then, after a linear layer transformation, the output z of the network is obtained h This step can be expressed as

[0070] x c =x⊙m+x h ⊙(1-m)

[0071] z h =Linear(x c )

[0072] Step ③: Generate weight γ according to the current time interval δ x , used to balance the input z h and historical information x h Then, γ x The concatenated missing marker matrix m is transformed to the same matrix as x through a linear transformation and a Sigmoid (·) activation function. h and z h The same size, denoted as β, is used to merge the features of Step ① and Step ②. The merging formula is

[0073] c h =β·x h +(1-β)·z h

[0074] Step ④: Change the c in Step ③ h After observation value replacement, it is expressed as c c . c Concatenate with the missing token m in the feature dimension as the input X of the GRU i , and the H of the previous layer t-1 Input into GRU to get H of the current timestamp t .

[0075] Next, the generation of time decay factor and GRU Cell are introduced in detail:

[0076] The process of time decay factor generation can be used to calculate the time decay factor γ according to the time interval matrix δ x , the time decay vector is introduced into the input update process, so that according to the time interval δ from the previous timestamp x, the features closer to the current timestamp are weighted higher, and the features farther away are weighted lower (Che et al, 2018). The present invention uses the self-attention mechanism to enhance the network input's perception of the time interval, and improves the ability to weigh the hidden variables and input weights of the GRU. The basic structure is as follows Figure 3 As shown, Figure 3 The overall input is δ, which is the time interval vector of multi-sensor data for each timestamp; the output is the time decay factor γ x . Figure 3 There are two parts in it. The left side is a basic linear layer, which is used to generate a temporary γ. The calculation method is

[0077] γ=ReLU(δW d ⊙I+b d )

[0078] Where W d and b d are the weights and biases of the network used to provide the attenuation factor; I is the unit matrix used to maintain the time interval information; ReLU(·) is the activation function. Figure 3 The right side of is the self-attention mechanism. First, the time interval δ is input into three linear layers in sequence. The outputs of the three linear layers are used as Query, Key and Value respectively. Then Q and K are dot-producted and divided by the scaling factor Avoid dot product results that are too large. Then normalize through Softmax(·), and the result obtained is the weight parameter of the self-attention mechanism. Dot product operation is performed on the weight parameter and V, and converted to the same size as the output of the basic linear layer through linear transformation. Finally, add the results on the left and right sides, and adjust the distribution range of the results through an exponential function to form the final γ x .

[0079] In GRU, the hidden state H t-1 and input X i GRU contains a reset gate R t and an update gate Z t The purpose of the reset gate is to calculate the candidate hidden state The purpose of the update gate is to act as a weight to adjust the newly generated H t Middle, H t-1and The structures of both are two fully connected layers using Sigmoid (·) activation function, expressed as

[0080] R t =Sigmoid(X i W xr +H t-1 W hr +b r )

[0081] Z t =Sigmoid(X i W xz +H t-1 W hz +b z )

[0082] Where W xr , W hr and b r is the network parameter of the reset gate, W xz , W hz and b z is the network parameter of the update gate. Then, the reset gate is integrated with the hidden state to calculate the candidate hidden state The formula is

[0083]

[0084] Where W xh , W hh and b h is the network parameter for calculating the candidate hidden state; ⊙ is the element-wise product operator. The calculation method for the GRU output hidden state is

[0085]

[0086] The PCHIP algorithm can be used for smooth interpolation of data. The implementation idea of ​​the algorithm is to construct a cubic polynomial in the interval between every two observation timestamps, and make these polynomials have the same numerical value and derivative on the observations. The algorithm includes four steps: split the original input to provide incomplete sequence data; calculate the derivative of each existing point in the data, and ensure that the derivatives of the data points on the left and right sides are consistent; construct a cubic Hermite polynomial in each interval, and ensure that the same function value and derivative value are at the position corresponding to each numerical value; use the cubic polynomial to calculate the interpolation result at any position. The PCHIP unit is further explained below: (1) Input data construction: Take the original data and real time interval data x, s provided above as input, and calculate the corresponding PCHIP interpolation result for each dimension to obtain another set of complete numerical values. For the data of the dth dimension, first give a set of incomplete data Xd , and the real time interval are sequentially spliced ​​to obtain the input of PCHIP, that is, (2) Calculating derivatives: Calculating Although the derivative of the value can be directly calculated by d = △y / △x, this calculation method cannot guarantee smoothness and monotonicity for the PHCIP interpolation method. In PCHIP, the following method is used to determine the derivative d of any k-th segment (i.e., between k and k+1): k .

[0087] First calculate the basic slope m k Let h k =s k+1 -s k , then in s k The basic slope at Then calculate d k . When k≠1 and k≠T, if m k =0, m k-1 =0 or SGN(m k )≠SGN(m k-1 ) holds true, then let d k =0; otherwise according to h k and h k-1 For m k and m k-1 Perform weighted harmonic mean calculation d k , the calculation method is

[0088] w1=2h k +h k-1

[0089] w2=h k +2h k-1

[0090] 1 / d k =1 / (w1+w2)*(w1 / m k +w2 / m k-1 )

[0091] Where SGN(·) represents the sign of the value, which can be -1, 0, or 1.

[0092] When k = 1 or k = T, it is necessary to use the one-sided three-point derivative estimate method to determine the boundary point derivative value. When k = 1, use h k ,h k+1 , m k , m k+1 Calculate d kFirst, get d k The initial estimate of

[0093] d k =G(h k ,h k+1 , m k , m k+1 )=((2×h k +h k+1 )×m k ) / (h k +h k+1 )

[0094] In order to ensure that the derivative does not destroy the shape and monotonicity of the interpolation function, it is necessary to adjust the derivative value under some conditions. First, determine SGN(d k )=SGN(m k ), if not satisfied, directly set d k =0; otherwise, judge SGN(m k )≠SGN(m k+1 ) and |d k |>3×|m k |Is it true? If true, then d k =3×m k When k = T, d k =G(h k-1 ,h k-2 , m k-1 , m k-2 ).

[0095] In this way, the derivative corresponding to each point can be obtained, and a sequence is formed according to the timestamps of different observations, which is represented by d.

[0096] (3) Constructing cubic Hermite polynomial

[0097] First, the interpolation parameters are calculated, taking into account the relationship between the derivative and the slope.

[0098] t k =(d k +d k+1 -2×m k ) / h k

[0099] The Hermite interpolation method also uses a cubic polynomial for interpolation. For any [x i , x i+1 ], the polynomial is in the form of

[0100] F k (x) = pk (xx i ) 3 +q k (xx i ) 2 +r k (xx i )+s k

[0101] The calculation method is p k =t k / h k ,q k =(m k -d k ) / h k -t,r k =d k ,s k =y k .

[0102] 1) Calculate interpolation results

[0103] Finally, according to the provided position in a corresponding interval such as x p ∈[x k , x k+1 ], select F k (x) can be based on the provided x p Find the corresponding interpolation result.

[0104] In the training process of the traditional RNN-based filling method, the error measurement between the reconstruction result of the neural network and the observation value is usually used as the loss function. The loss function of the model training process of the method proposed in the present invention includes four parts: the reconstruction error of each of the two Interval-aware Temporal Imputation Networks, the filling result error between the two Interval-aware Temporal Imputation Networks, and the error between the average value of the filling results of the two Interval-aware Temporal Imputation Networks and the filling result of the PCHIP unit. After the parameter training of the two models is completed, the test data is input into the model to obtain two sets of forward and reverse reconstruction results, and the reconstruction results are added and averaged accordingly as the final filling result.

[0105] First, the reconstruction errors of the two Interval-aware Temporal Imputation Networks are respectively. For each model, the reconstruction error consists of three parts: x h,z h ,c h Compared with the MAE of x, this loss needs to be calculated at each timestamp and accumulated after all timestamps are calculated. The calculation at each timestamp can be expressed as

[0106]

[0107] Where SUM(·) represents the sum of all numbers in the matrix. The reconstruction loss of each timestamp is added together as the loss function of the final unidirectional RNN network, expressed as The forward filling network can be expressed as The backfill network can be expressed as The second is the filling result error between the two Interval-aware Temporal Imputation Networks, that is, the error of the filling result consistency. This error needs to be scaled to 10% of the original value due to its large value. The calculation process can be expressed as Finally, the error between the filling result and the PCHIP filling result can be expressed as The final loss function of the network training can be expressed as

[0108]

[0109] Example:

[0110] Taking aircraft engines as an example, a time interval perception network with piecewise cubic Hermite interpolation constraints is proposed for aviation equipment life cycle degradation data interpolation method:

[0111] The first is the acquisition of raw data. This step is used to extract part of the data from the engine's operating data to form a data set. This example uses two data sets, namely C-MAPSS data and real civil aviation engine data. The two data sets have the same data format, both of which are multivariate time series data of unequal length and containing missing values. The data is first collected by sensors installed on the aircraft engine, and then the aircraft's Quick Access Recorder (QAR) records the complete data of all stages, and uses ACARS as a channel to send key data of key stages to the ground in real time. Since the data of each flight will form a long data containing various stages, first select a key state that can reflect the engine performance, and then extract the characteristic values ​​under this state to form a timestamp data, and then splice all flights of a single engine in chronological order to obtain the operating status data of each engine;

[0112] The next step is the data preprocessing stage: the first step is to split the data set, divide multiple engines into different data sets according to the engine number, take 20% of the entire data set as the test set, and take 20% of the training set as the validation set; the second step is to manually mask 10% of the data to verify the model effect and test the filling effect of different methods; the third step is to splice the original data sequence according to the training set, validation set and test set, use the training set data as the standard, standardize and normalize, weaken the data difference in the feature dimension, and improve the filling effect of the neural network; the fourth step is to use a sliding window to cut the data of different lengths into the same length for easy input into the neural network;

[0113] The third step is to train the imputation network. During the training process, the incomplete training set is first input into two Interval-aware Temporal Imputation Networks to obtain the reconstructed data and temporary imputed data output by the network. Then, the MAE of the reconstructed data and the original data is used as part of the loss function, and the MAE of the consistency of the imputation results of the two networks is used as part of the loss function. The two groups of imputation results are averaged and used as the temporary imputed data in the training stage. Subsequently, the incomplete data is input into the PCHIP unit to form temporary imputed data. The MSE of the imputed data and the imputation results of the PCHIP unit are used as part of the loss function to jointly optimize the network parameters of the two Interval-aware TemporalImputation Networks.

[0114] Finally, the application of the filling network: first, the test data is input into the two trained Interval-aware Temporal Imputation Networks to obtain the reconstructed data and temporary filling data, and then the temporary filling results of the two networks are averaged to be used as the final filling result of the test set. Finally, the difference between the filling result of the test set and the original data can be measured. The measurement indicators are MSE and MAE, and the calculation methods are

[0115]

[0116] Next is the conduction of the experiment and the analysis of the experimental results. It mainly consists of three parts. The first part introduces the dataset and experimental platform; the second and third parts are the filling results and analysis of the C-MAPSS dataset and the real aircraft engine dataset respectively.

[0117] The C-MAPSS dataset is a set of public datasets related to engine health generated by the C-MAPSS simulator and used to simulate the degradation trend of turbofan engines (Saxena et al., 2008). The dataset contains four subsets, namely FD001~FD004, and the details of the dataset are shown in Table 2. Each dataset contains the engine number, cycle number, three operating states, and 21 sensor data.

[0118] Table 2 Details of C-MAPSS dataset

[0119]

[0120] In order to avoid the influence of different operating states, this experiment selects FD001 and FD003 for the experiment. At the same time, in order to avoid the influence of some data with less correlation with engine state changes on the experimental results, the Pearson correlation coefficient method is used to eliminate the sensor data with less correlation with the experimental results. The remaining data is shown in Table 3. The original missing rate of the experimental data is set to 10%. On this basis, 10% is set to fill the network training and effect verification.

[0121] Table 3 Details of sensor data used in the experiment

[0122]

[0123] The real aircraft engine data set comes from the turbofan engine operation data of an Asian airline. The basic structural diagram of the turbofan engine is shown in Figure 5, which mainly includes high and low pressure turbines (HPT, LPT), combustion chambers (CC), high and low pressure compressors (HPC, LPC) and fans. The data used in the present invention contains four dimensions, N1 is the low pressure rotor speed, N2 is the high pressure rotor speed, FF is the fuel consumption, and the thermocouple on the T49.5 harness assembly can be used to collect the engine exhaust temperature EGT.

[0124] The actual data contains faulty data that may affect the data regularity. Since the present invention does not involve fault diagnosis, in order to avoid the influence of faulty data, only more than 10 samples before the fault occurs are collected as the data set. The preset missing rate and manually masked data percentage of this data set are consistent with the C-MAPSS data set, both of which are 10%.

[0125] The Python version used in the experiment is 3.10, the PyTorch version is 2.1.1, the machine CPU is Intel Core i7-9700K, and the memory is 16G DDR4. In this experiment, the step size of the two data sets is 48, the number of features of the C-MAPSS data set is 17, and the number of features of the real civil aviation engine data set is 4; the GRU hidden layer size is 128; the mini-batch size is 32. The training is performed for a maximum of 30 epochs. If the loss does not decrease within 3 consecutive epochs, the training is stopped in advance; the network optimizer uses Adam, and the initial learning rate is 0.001. The experimental results of the real aircraft engine data set are analyzed as follows:

[0126] Considering that different data sets should use different hyperparameters, in the process of conducting experiments based on real aircraft engine data, the step size is set to 0.6 under the condition of α∈[0,6), and ten experiments are conducted for each α to take the average and median. The MSE and MAE of the experimental results are shown in Tables 4 and 5. As can be seen from Table 4, MSE has a minimum average value of 0.2590 at α=0.6 and a minimum median of 0.2456 at α=1.2; as can be seen from Table 5, MAE has a minimum average value of 0.3951 at α=0.6 and a minimum median of 0.3857 at α=1.2. Since this example focuses on the rationality of the filling results, it is more inclined to the results with smaller overall loss. In this example, α=0.6 is selected as the hyperparameter on the real aircraft engine data set.

[0127] Table 4 Relationship between α and MSE on the civil aviation engine dataset

[0128]

[0129] Table 5 Relationship between α and MAE on the civil aviation engine dataset

[0130]

[0131] In order to evaluate the effect of the proposed method, CITIN is compared with different methods proposed in recent years, such as CSDI, GP-VAE, and BRITS. The error of ten repeated experiments is as follows: Figure 6 As shown. Figure 6 As can be seen in (a), the overall MSE value of the CSDI result is relatively large. Figure 6 In (b), it can be seen that GP-VAE has a larger value under the MAE evaluation index. In the 10 experimental results, CITIN is better than the other three methods from the perspective of MSE or MAE.

[0132] Tables 6 and 7 show the average, standard deviation and median of MSE and MAE of ten experimental results. It can be seen from Table 6 that the MSE and MAE of the proposed method are the smallest in average value, which are 0.0717 and 0.1949 respectively, and the standard deviations of MSE and MAE of ten repeated experiments of the proposed method are also small, which are 0.0022 and 0.0029 respectively, which shows that the overall filling effect of the proposed method is better, and the filling results of multiple experiments are relatively stable. It can be seen from Table 7 that the MSE and MAE of the proposed method are 0.0722 and 0.1951 respectively, which are better than the methods of the present invention, indicating that the proposed method performs more consistently in most cases.

[0133] Comparative experiments of different methods:

[0134] Table 6 Average results on the C-MAPSS dataset

[0135]

[0136] Table 7 Median results on the C-MAPSS dataset

[0137]

[0138] Hypothesis Validity Experiment:

[0139] As mentioned above, the basic principle of the proposed method is to reduce the possibility of generating unreasonable values ​​by compressing the deviation between the filled value and the original data distribution. Therefore, it is necessary to verify the deviation between the filled value and the actual value compared with not using statistical methods to constrain the range. For the convenience of comparison, BRITS, which has a better effect among the comparison methods, is selected as the control of the proposed method. Table 8 shows the ten experimental results and the average values ​​of the range of the difference between the filled value and the actual value of BRITS and the proposed method. The left side of the table is the left and right boundaries of BRITS, and the right side is the left and right boundaries of the proposed method. It can be seen from Table 8 that the average left boundary of the difference between the filled results and the actual value of BRITS in ten experiments is -1.4613, the right boundary is 1.9997, and the average interval of the left and right boundaries is 3.4610; in comparison, the left and right boundaries of the proposed method are -1.2999 and 1.8036 respectively, and the interval length of the left and right boundaries is 3.1035. Therefore, the left and right boundaries of the proposed method are smaller than the absolute value of BRITS. Experimental results show that the error range between the filled value and the actual value of the proposed method is smaller.

[0140] Table 8. Deviation left and right boundaries of BRITS and the proposed method on the C-MAPSS dataset

[0141]

[0142] Experimental results under different signal-to-noise ratios

[0143] Figure 7 The experimental results comparison of CITIN and BRITS under different signal-to-noise ratios are shown. Figure 7 (a) is a comparison of MSE under different signal-to-noise ratios. It can be seen from the figure that when the signal-to-noise ratio is 10dB, the MSE of both methods exceeds 0.1, and does not exceed 0.1 under other signal-to-noise ratios. However, the MSE of CITIN under various signal-to-noise ratio conditions is smaller than that of BRITS, and the standard deviation of the ten experiments is also smaller. Figure 7 The MAE comparison experiment in (b) shows that the error of CITIN is smaller than that of BRITS under various signal-to-noise ratio conditions, and the standard deviation of CITIN is significantly smaller than that of BRITS under various noise conditions in the experiment. The above results show that CITIN works better and is more stable under different levels of noise.

[0144] Analysis of experimental results on real aircraft engine dataset

[0145] Selecting effective weight coefficients: Considering that different data sets should use different hyperparameters, in the process of conducting experiments based on real aircraft engine data, the step size is set to 0.6 under the condition of α∈[0,6), and ten experiments are conducted for each α to obtain the average and median. The MSE and MAE of the experimental results are shown in Tables 9 and 10. As can be seen from Table 9, MSE has achieved the minimum average value of 0.2590 at α=0.6 and the minimum median of 0.2456 at α=1.2; as can be seen from Table 10, MAE has achieved the minimum average value of 0.3951 at α=0.6 and the minimum median of 0.3857 at α=1.2. Since this example focuses on the rationality of the filling results, it is more inclined to the results with smaller overall loss. In this example, α=0.6 is selected as the hyperparameter on the real aircraft engine data set.

[0146] Table 9 Relationship between α and MSE on the civil aviation engine dataset

[0147]

[0148] Table 10 Relationship between α and MAE on the civil aviation engine dataset

[0149]

[0150] Comparative experiments of different methods

[0151] Figure 8The experimental results of the four methods on the real civil aviation engine data set are compared. It can be observed from the figure that the error of CSDI is the largest among the four methods in terms of MSE and MAE. This may be due to the small amount of real aircraft engine data and the failure of CSDI to fully converge due to the method itself. In most experimental results, the MSE and MAE of CITIN are smaller than those of GP-VAE and BRITS. In the third experiment, the MAE of CITIN is greater than that of CSDI; in the sixth experiment, the MSE and MAE of CITIN are greater than those of BRITS. One possible reason is that in these experiments, the samples selected for training cannot represent the general phenomenon of the engine degradation process.

[0152] Tables 11 and 12 show the mean and standard deviation of the MSE and MAE errors of the four methods in ten repeated experiments, as well as the corresponding median. As can be seen from Table 11, the mean values ​​of the MSE and MAE of the ten experiments of CITIN are 0.2590 and 0.3951, while the standard deviations of the ten experiments are 0.0612 and 0.0515, respectively, which are slightly larger than 0.0631 and 0.0455 of BRITS. This may be due to the fact that the noise of the dataset itself is larger than that of the C-MAPSS dataset, which makes the results of CITIN more different in multiple experiments. As can be seen from Table 12, the median of the proposed method is also the smallest among all the compared methods, with MSE and MAE of 0.2615 and 0.4101, respectively.

[0153] Table 11 Average results on the civil aviation engine dataset

[0154]

[0155] Table 12 Median results on the civil aviation engine dataset

[0156]

[0157] Hypothesis validity experiment: In order to verify whether CITIN can reduce the possibility of generating unreasonable values, it is necessary to compare CITIN with other advanced methods. For convenience, the experiment directly uses BRITS as a control. The deviation between the filling results and the actual values ​​is shown in Table 13. As with the experiment of the CMAPSS dataset, the filling values ​​of different methods are subtracted from the actual values, and the left boundary, right boundary and deviation range of the deviation values ​​of multiple experiments are recorded in the table. It can be seen from Table 13 that the absolute values ​​of the average left boundary and the average right boundary of CITIN's ten experiments are 1.6312 and 2.0151, respectively, which are smaller than the absolute values ​​of the left and right boundaries of BRITS, 1.6418 and 2.1647, and the average value of the overall deviation range is also smaller.

[0158] Table 13 The left and right boundaries of the deviations of BRITS and the proposed method on the real civil aviation dataset

[0159]

[0160] Experimental results under different signal-to-noise ratios:

[0161] Fig. 9 The MSE and MAE data of the infill values ​​and actual values ​​of real civil aviation data under different signal-to-noise ratios are shown. It can be seen from the figure that although the standard deviations of the two methods are close under different signal-to-noise ratios, CITIN has a smaller error than BRITS under the same signal-to-noise ratio.

[0162] The present invention proposes a constrained time interval perception time series interpolation network for complex equipment such as aircraft engine performance data. The network not only considers the impact of the engine's performance degradation at different stages and different durations of each flight, but also more accurately fills in the missing parts of the engine performance data; at the same time, in view of the problem that unreasonable values ​​sometimes appear in the neural network due to its unexplainability, PCHIP is used to constrain the range of the filling results generated by the neural network during the training phase of the filling network, thereby reducing the possibility of ITIN generating unreasonable values. The present invention uses C-MAPSS data sets and real civil aviation engine data sets to carry out experiments. The experimental results show that compared with the current more advanced methods, the method proposed in the present invention has achieved better results. In addition, the present invention also discusses the hyperparameter values ​​of the proposed method, and experiments on the stability of the proposed method under different noises of the data set. The experimental results show that the stability of the proposed method is better than that of the more advanced RNN-type methods. The method proposed in the present invention can not only be applied to more accurately model the missing data filling process of aircraft engines, but also be applicable to other types of complex equipment degradation data filling; in addition, the paradigm of the present invention using statistical methods to constrain neural networks can be applied to the combination of other neural networks and statistical methods.

Claims

1. A complex equipment data processing method based on CITIN, characterized in that: The following steps are involved: Step 1: Get the original data; Step 2: Data preprocessing: split the data set to obtain the test set and training set, splice the original data sequence according to the training set, validation set and test set, use the training set data as the standard, standardize and normalize, weaken the data differences in the feature dimension, improve the filling effect of the neural network, and use the sliding window to cut the data of different lengths into the same length for easy input into the neural network; Step 3: Establish a filling network model and train it. The filling network model introduces Transformer to process the time interval information based on GRU, and combines the result with the missing label matrix to adjust the hidden state of the missing part and the input weight to form a time interval-aware temporal interpolation network. Secondly, the piecewise cubic Hermite interpolation polynomial PCHIP is introduced into the training process of the time interval-aware temporal interpolation network to constrain the range of filling result generation; During training, the incomplete training set is first input into two time interval-aware temporal interpolation networks to obtain the reconstructed data and temporary filling data output by the networks. Then, the MAE of the reconstructed data and the original data is used as part of the loss function, and the MAE of the consistency of the filling results of the two networks is used as part of the loss function. The two groups of filling results are averaged as the temporary filling results in the training stage. Subsequently, the incomplete data is input into the PCHIP unit to form temporary filling data. The MSE of the temporary filling results and the filling results of the PCHIP unit are used as part of the loss function to jointly optimize the network parameters of the two time interval-aware temporal interpolation networks. Step 4: Application of the filling network: First, input the test data into the two time interval-aware time series interpolation networks that have been trained to obtain reconstructed data and temporary filling data. Then, the temporary filling results of the two networks are averaged to be used as the final filling result of the test set.

2. The complex equipment data processing method based on CITIN according to claim 1 is characterized in that: The input of the filling network model includes incomplete data X, missing label matrix M, time interval matrix Δ and absolute time s, multivariate time series is incomplete data, where T is the length of the data in the time dimension, D is the feature dimension of the data, and the missing marker matrix M is the position where the data of the original data X is missing. The shape is the same as X. For the data of the dth dimension at time t, have θ is represented as the time interval matrix of X, with the same shape as X. The time interval matrix θ is calculated separately for each dimension, and the calculation rules for the corresponding position include three: first, the time interval δ1 of the first timestamp is set to 0; Secondly, for the subsequent timestamp t, if there is data in the sample sequence X of the dimension corresponding to the previous timestamp of the timestamp, the current absolute time is subtracted from the absolute time of t-1 as the value corresponding to Δ; finally, if the data of x at the corresponding position of t-1 is empty, on the basis of subtracting the current absolute time from the absolute time of t-1, the time interval value of the previous timestamp is also added. The calculation of Δ in the dth dimension at time t is expressed as Among them, s t represents the absolute time at the tth moment; Represents the time interval matrix of the dth dimension at time t.

3. The complex equipment data processing method based on CITIN according to claim 1 is characterized in that: The input data of the padding network model is labeled as The positions of these data are marked using the indicator matrix I. The shape of the indicator matrix is ​​the same as the original data. The data positions where the observed values ​​exist but are hidden in actual use for prediction are represented as 1, and the remaining positions are represented as 0. The indicator matrix I at time t is represented by the dth dimension:

4. The complex equipment data processing method based on CITIN according to claim 3 is characterized in that: The filling network model includes two time interval-aware time series interpolation networks, one of which is used to capture the law from the forward direction of the training data, and the other is used to capture the law from the reverse direction. The two networks each provide a temporary filling result, and a PCHIP unit is also provided for calculating the filling result using the statistical method. The inputs of the two time interval-aware time series interpolation networks include the original data X, the time interval matrix Δ and the missing mark matrix M. The first time interval-aware time series interpolation network INIT1 starts from the first timestamp and processes all the features of each timestamp in sequence until the last one, representing the process of forwardly capturing the data relationship in the time dimension, and the hidden layer H of the INIT structure is processed. t Input to the next layer, and convert the output c1~c T Temporary imputed data1 is spliced ​​together according to the time dimension. Expressed as C=Concat(c1,c2,…,c T ), Where C represents the matrix after the output of each step of the GRU model is concatenated; Concat(·) represents concatenating vectors into matrices, ⊙ represents multiplication of corresponding positions of matrices of the same size, and the second time interval-aware temporal interpolation network INIT2 has the opposite processing process to the first one. It learns the time series from back to front in the time dimension to capture the impact of future data on the filling results of past data. The filling result of the second time interval-aware temporal interpolation network INIT2 is expressed as The input of the PCHIP unit processing flow includes the original data X and the absolute time vector s corresponding to each timestamp. The PCHIP unit needs to process each dimension of X separately. When processing the data of the dth dimension, The derivative is calculated for each point of , and then each coefficient of the cubic function is calculated based on the derivative and the slope, and finally the position s containing the missing data is t Substitute the Hermite polynomial composed of multiple functions and solve the corresponding position After all missing values ​​are filled, the filling results can be obtained 5. The complex equipment data processing method based on CITIN according to claim 4 is characterized in that: The time interval-aware time series interpolation network processes all feature dimension data at each moment. In addition to the original input of each timestamp, it also contains the information transmitted by the historical timestamps, where H t-1 Represents the hidden features transmitted by the previous timestamp. When processing the first timestamp, H transmitted by the historical timestamp t-1 is set to 0. The following is the specific process of processing each timestamp data: Step ①: The hidden variables of the previous layer are converted to x after a linear layer dimension reduction. h , x h The size of is the same as the input x; Step ②: Use the existing value in x to replace the signal attenuation of the features of the hidden layer of the previous layer x h Reconstruct the data at the corresponding position to form the input x c Then, after a linear layer transformation, the output z of the network is obtained h , expressed as x c =x☉m+x h ☉(1-m) z h =Linear(x c ) Step ③: Generate weight γ according to the current time interval δ x , used to balance the input z h and historical information x h The important weights of x The concatenated missing marker matrix m is transformed to the same matrix as x through a linear transformation and a Sigmoid (·) activation function. h and z h The same size, denoted as β, is used to merge the features of Step ① and Step ②. The merging formula is c h =β·x h +(1-β)·z h , Step ④: Change the c in Step ③ h After observation value replacement, it is expressed as c c , c c Concatenate with the missing token m in the feature dimension as the input X of the GRU i , and the H of the previous layer t-1 Input into GRU to get H of the current timestamp t .

6. The complex equipment data processing method based on CITIN according to claim 5 is characterized in that: In Step ③, the overall input is δ, which is the time interval vector of multi-sensor data for each timestamp; the output is the time attenuation factor γ x , the self-attention mechanism is introduced to enhance the perception of network input for time intervals, and improve the ability to weigh the hidden variables and input weights of GRU. The left side of the self-attention mechanism enhanced network is a basic linear layer, which is used to generate a temporary γ. The calculation method is: γ=ReLU(δW d ☉Ⅰ+b d ) Where W d and b d are the weights and biases of the network used to provide the decay factor; I is the unit matrix, which is used to maintain the existence of time interval information; ReLU(·) is the activation function. The right side of the self-attention mechanism enhanced network is the self-attention mechanism. First, the time interval δ is input into three linear layers in sequence, and the outputs of the three linear layers are used as Query, Key and Value respectively. Then Q and K are dot-producted and divided by the scaling factor To avoid the dot product result being too large, it is then normalized by Softmax(·). The result obtained is the weight parameter of the self-attention mechanism. The weight parameter is dot-producted with V and converted to the same size as the output of the basic linear layer through a linear transformation. Finally, the results on the left and right sides are added together, and the distribution range of the results is adjusted through an exponential function to form the final γ x .

7. The complex equipment data processing method based on CITIN according to claim 6 is characterized in that: In Step ③, in GRU, the hidden state H t-1 and input X i Going a step further, the GRU contains a reset gate R t and an update gate Z t , the purpose of the reset gate is to calculate the candidate hidden state The purpose of the update gate is to act as a weight to adjust the newly generated H t Middle, H t-1 and The weights of both structures are two fully connected layers using the Sigmoid (·) activation function, expressed as R t =Sigmoid(X i W xr +H t-1 W hr +b r ) Z t =Sigmoid(X i W xz +H t-1 W hz +b z ) Where W xr , W hr and b r is the network parameter of the reset gate, W xz , W hz and b z is the network parameter of the update gate, integrates the reset gate with the hidden state, and calculates the candidate hidden state The formula is Where W xh , W hh and b h is the network parameter for calculating the candidate hidden state; ⊙ is the element-wise product operator, and the calculation method for the GRU output hidden state is 8. The complex equipment data processing method based on CITIN according to claim 7 is characterized in that: The PCHIP unit is used for smooth interpolation processing of data, specifically: constructing a cubic polynomial in the interval between every two observation time stamps, and making these polynomials have the same numerical value and derivative on the observation value, including four steps: splitting the original input to provide incomplete sequence data; calculating the derivative of each existing point in the data, ensuring that the derivatives of the data point on the left and right sides are consistent; constructing a cubic Hermite polynomial in each interval, and ensuring that the same function value and derivative value are present at the position corresponding to each numerical value; using the cubic polynomial to calculate the interpolation result at any position.

Images