A clock error prediction method based on Kalman filtering combined with long short-term memory network

Abstract

The present invention discloses a clock error prediction method based on Kalman filtering combined with a long short-term memory network, belonging to the technical field of two-way comparison clock error prediction. The present invention comprises the following steps: constructing an LSTM network; constructing a KF algorithm model; obtaining training data and performing normalization processing; using the normalized training data to jointly train the LSTM network and the KF algorithm model; obtaining clock error data in actual scenarios and performing normalization processing; inputting the processed clock error data into the trained LSTM network, and using the output of the LSTM network as the parameter Q in the KF algorithm model. t and R t After running the KF algorithm model, the clock error prediction result is obtained. This invention optimizes the Kalman filter algorithm using a long short-term memory network, resolving the Kalman filter's reliance on prior knowledge. By combining the long-term memory capacity of the long short-term memory network with the real-time update characteristics of the Kalman filter algorithm, more accurate clock error prediction is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of two-way comparison clock error prediction in a time synchronization system, and specifically relates to a clock error prediction method based on a Kalman filter (KF) combined with a long short-term memory (LSTM) network. Background Art

[0002] Time synchronization plays a crucial role in modern science and technology, and clock error prediction is one of the key technologies for achieving precise time synchronization. Extensive research has been conducted in this area, and a variety of prediction models have been developed, including linear regression models, polynomial models, grey models, least squares support vector machines, and the KF algorithm. The KF algorithm is widely used due to its recursive nature, minimal data storage requirements, and applicability to multidimensional and non-stationary random processes.

[0003] The traditional KF algorithm uses a state-space model of signal and noise, using the estimated value at the previous moment and the observed value at the current moment to update the estimate of the state variable and calculate the estimated value at the current moment. This is of great significance for real-time data processing. However, the KF state matrix, state noise, and observation noise are all derived from prior knowledge and are influenced by human prior knowledge, which significantly limits the KF's computational capabilities.

[0004] As can be seen, although the KF algorithm has achieved widespread application in practice, its predictive performance is often limited by the accuracy and completeness of prior knowledge. In recent years, the LSTM network has attracted much attention for its outstanding time series prediction capabilities, providing a new approach to addressing the limitations of the KF algorithm. LSTM is a special type of recurrent neural network (RNN) designed to address the long-term dependency problem existing in RNNs, namely the inability to effectively learn features in longer time series. LSTM achieves this goal by introducing memory cells and three logic gates (input gate, forget gate, and output gate) to control information transmission. It is suitable for processing and predicting events in time series with relatively long intervals and delays.

[0005] For an LSTM network, signal control at time t is determined by the output at time t-1 and the input at time t. This involves the memory cell, forget gate, input gate, and output gate. The forget control determines whether the data stored in the memory cell is valid. When the forget gate is valid, the memory cell data is cleared. The input gate controls whether the input data at that moment is valid. The input data is calculated with the memory cell data after passing through the forget gate, and the output gate determines whether the data should be output. When the output gate is valid, the data at that moment is output.

[0006] However, there is no application in the prior art that combines the LSTM network with the KF algorithm model for clock error prediction. Summary of the Invention

[0007] In view of this, the present invention proposes a clock error prediction method based on Kalman filtering combined with long short-term memory network, which effectively solves the problem that the KF algorithm relies on prior knowledge, improves the accuracy and stability of clock error prediction, and thus enhances the performance of the time synchronization system.

[0008] The technical solution adopted in the present invention is:

[0009] A clock error prediction method based on Kalman filtering combined with long short-term memory network includes the following steps:

[0010] Step 1: Build an LSTM network, including input layer, hidden layer and output layer;

[0011] Step 2: Construct a KF algorithm model. The first equation of the KF algorithm model is used to derive the state prediction at time t+1 from the state estimate at time t. The second equation of the KF algorithm model is used to adjust the prediction value at time t+1 to obtain the state estimate at time t+1.

[0012] Step 3: Obtain the measured clock error data, extract a portion of the measured clock error data as training data, and merge the training data into a 1-row n-column data matrix x t , n is the length of training data, and normalization is performed;

[0013] Step 4: Use the normalized training data to jointly train the LSTM network and KF algorithm model;

[0014] Step 5: Get the clock error data in the actual scene and merge it into a 1-row and m-column data matrix x' t , m is the length of clock error data, and normalization is performed;

[0015] Step 6: Input the data processed in step 5 into the trained LSTM network, and use the output of the LSTM network as the parameter Q in the KF algorithm model. t and R t , after the operation of the KF algorithm model, the clock error prediction result is obtained.

[0016] Furthermore, in step 1, the input layer includes 1 node, the hidden layer includes 4 nodes, and the output layer includes 2 nodes. The change relationship of each node in the LSTM network is as follows:

[0017] f t =s(W fh ×h t-1 +W fxx t +b f ) (1)

[0018] i t =s(W ih h t-1 +W ix x t +b i ) (2)

[0019] o t =s(W oh h t-1 +W ox x t +b o ) (3)

[0020]

[0021] Among them, x represents the input data of the current node, σ represents the sigmoid activation function, tanh is an activation function, W is the weight matrix of each gate, the subscript of W represents two corresponding parameters, and b is the bias term of each gate. represents element-wise multiplication, f represents the forget gate parameter, i represents the input gate parameter, and o represents the output gate parameter. Represents the current memory unit state, c represents the current memory unit output, h represents the output value of the current node, and the subscripts t and t-1 represent two adjacent moments.

[0022] Furthermore, the first equation of the KF algorithm model is:

[0023]

[0024] Among them, F t Indicates the operating rules of the system itself. represents the state estimate at time t, represents the state prediction at time t+1;

[0025] The second equation of the KF algorithm model is:

[0026]

[0027] in, represents the state estimate at time t+1, Y t+1 represents the data observation value at time t+1, H t+1 represents the observation matrix, G t+1 is the Kalman gain, and its calculation formula is:

[0028]

[0029] Among them, Rt+1 represents the variance of the observation noise, P t+1 / t It represents the prediction error of the data at time t+1 predicted by the data at time t, and its calculation method is:

[0030] P t+1 / t =F t P t / t F t T +Q t (10)

[0031] Among them, Q t represents the variance of the state noise, P t / t It represents the estimation error of the estimated value at time t estimated by the data at time t;

[0032] The estimation error of the estimated value at time t+1 estimated by the data at time t+1 is:

[0033]

[0034] Furthermore, the specific method of step 4 is:

[0035] Step 401: training data x t Input the LSTM network and use the output of the LSTM network as the parameter Q of the KF algorithm model t and R t ;

[0036] Step 402: Q t and R t Two parameters are input into the KF algorithm model and calculated according to formula (7) to formula (11) to obtain the Kalman estimation value That is the calculation result of formula (7);

[0037] Step 403: Calculate the error term of the LSTM network according to formula (12):

[0038]

[0039] Step 404: Feedback the error term error to the LSTM network to update the weight matrix;

[0040] Step 405: Perform multiple rounds of iterations to complete the joint training of the LSTM network and the KF algorithm model.

[0041] Furthermore, in step 6, the operation of the KF algorithm model refers to the calculation according to formulas (7) to (11), and the obtained clock error prediction result is the calculation result of formula (7).

[0042] The present invention has the following beneficial effects:

[0043] 1. By combining the long-term memory capability of the LSTM network and the real-time update characteristics of the KF algorithm, the present invention can achieve more accurate clock error prediction and improve the accuracy and stability of clock error prediction.

[0044] 2. The present invention helps to improve the performance of the time synchronization system and provides new methods and ideas for research in related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 Flowchart of the clock error prediction method in an embodiment of the present invention.

[0046] Figure 2 Schematic diagram of iterative training error of the clock error prediction method in an embodiment of the present invention.

[0047] Figure 3 Schematic diagram comparing the prediction results of clock error data using different methods in an embodiment of the present invention.

[0048] Figure 4 Schematic diagram of the Q and R prediction results output by the clock error prediction method in an embodiment of the present invention.

[0049] Figure 5 Schematic diagram of the comparison between the predicted value and the measured value of the KF algorithm model.

[0050] Figure 6 Schematic diagram of the comparison between the predicted value and the measured value of the LSTM algorithm model.

[0051] Figure 7 This is a schematic diagram comparing the predicted values ​​and measured values ​​of the LSTM+KF algorithm model. DETAILED DESCRIPTION

[0052] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0053] A clock error prediction method based on Kalman filtering combined with long short-term memory network, such as Figure 1 As shown, the following steps are included:

[0054] Step 1: Build an LSTM network, including an input layer, a hidden layer, and an output layer. Specifically, the input layer includes 1 node, the hidden layer includes 4 nodes, and the output layer includes 2 nodes. The change relationship of each node in the LSTM network is as follows:

[0055] f t =s(W fh ×h t-1 +W fx x t +b f ) (1)

[0056] it =s(W ih h t-1 +W ix x t +b i ) (2)

[0057] o t =s(W oh h t-1 +W ox x t +b o ) (3)

[0058]

[0059] Among them, x represents the input data of the current node, σ represents the sigmoid activation function, tanh is also an activation function, W is the weight matrix of each gate, and b is the bias term of each gate. represents element-wise multiplication, f represents the forget gate parameter, i represents the input gate parameter, and o represents the output gate parameter. Represents the current memory unit state, c represents the current memory unit output, and h represents the output value of the current node.

[0060] The training method of LSTM is the same as that of RNN, which uses the backpropagation method. First, based on equations (1) to (6), the output value of each neuron is forward calculated, and then the error term of each neuron is calculated and backpropagated. The error calculation formula is:

[0061]

[0062] According to the calculated error value, backpropagation is performed to obtain the gradient of each weight, and the weight matrix is ​​updated. The training of the LSTM network is finally completed through iterations.

[0063] Step 2: Build the KF algorithm model, specifically:

[0064] The first equation of the KF algorithm model is to derive the state prediction at time t+1 from the state estimation at time t:

[0065]

[0066] Among them, F t Indicates the operating rules of the system itself. represents the state estimate at time t, Represents the state prediction at time t+1. Adjusting the predicted value at time t+1 gives the state estimate at time t+1, which is the second equation of the KF algorithm model:

[0067]

[0068] in, represents the state estimate at time t+1, Y t+1 represents the data observation value at time t+1, H t+1 represents the observation matrix, G t+1 Kalman gain, calculated as:

[0069]

[0070] Among them, R t+1 represents the variance of the observation noise, P t+1 / t It represents the prediction error of the data at time t+1 predicted by the data at time t, and its calculation method is:

[0071] P t+1 / t =F t P t / t F t T +Q t (10)

[0072] Among them, Q t represents the variance of the state noise, P t / t It represents the estimation error of the estimated value at time t estimated by the data at time t, then the estimation error of the estimated value at time t+1 estimated by the data at time t+1 is:

[0073]

[0074] Step 3: Obtain the clock error data in the actual scene, intercept a part of the measured data as training data, and merge it into a 1-row n-column data matrix x t (n is the length of training data) and normalize it;

[0075] Step 4: Use the training data to jointly train the LSTM network and KF algorithm model. The specific method is as follows:

[0076] Step 401: training data x t Input the LSTM network and use the output of the LSTM network as the parameter Q of the KF algorithm t and R t ;

[0077] Step 402: Q t and R t The two parameters are input into the KF algorithm and calculated according to equations (7) to (11) to obtain the Kalman estimate

[0078] Step 403: Calculate the error term of the LSTM network according to formula (12):

[0079]

[0080] Step 404: Feedback the error term error to the LSTM network to update the weight matrix;

[0081] Step 405: Perform 50 iterations to complete the training of the LSTM+KF algorithm.

[0082] Step 5: Obtain the clock error data in the actual scene. Except for the training data intercepted in step 3, all of them are used as test data and merged into a data matrix x' with 1 row and m columns. t (m is the length of the test data), and normalize it;

[0083] Step 6: Test data x' t Input the trained LSTM network and calculate the parameter Q in the output KF algorithm model t and R t , and then through the KF algorithm model operation of formula (7) to formula (11), the Kalman estimation value is obtained That is, the clock error prediction result obtained by the LSTM+KF algorithm.

[0084] Since the KF algorithm relies heavily on prior knowledge, namely the state matrix F, state noise V t and the observation noise W t Therefore, this method uses the LSTM network to optimize the KF algorithm so that it does not require the input of prior knowledge and can be used directly, and has stronger linear fitting and data prediction capabilities.

[0085] The basic idea of ​​LSTM+KF algorithm is to use LSTM network training to predict the prior knowledge that cannot be directly obtained in KF. Generally speaking, it is assumed that the input noise is Gaussian white noise with mean 0 and variance Q t and R t In addition, the state matrix F and the transfer matrix H can be considered as the unit matrix in the clock error prediction application, that is, F = H = I. The modified LSTM + KF algorithm process is as follows Figure 1 shown.

[0086] like Figure 1 As shown in (a), the KF parameter Q is obtained by LSTM network training t and R t , and then substitute it into the KF algorithm to estimate the clock error data. Then, the error term error is calculated according to formula (14), and the error is fed back to the LSTM network to update the weight matrix. After successive iterations, the LSTM+KF algorithm model is established.

[0087] like Figure 1 As shown in (b), for the trained LSTM+KF algorithm model, the clock error data is input into the LSTM network to calculate the KF prior parameter Q t and R t , then use the KF algorithm to estimate and predict the clock error and output the prediction result

[0088] Next, we select the clock difference data obtained by measuring using a time synchronization system based on two-way comparison. The measurement method is to turn off the time synchronization algorithm program and only measure the time difference between the clock signal transmitted from the near end to the far end and the local clock signal at the far end. The measurement results are used to perform clock difference prediction simulation.

[0089] The number of LSTM network nodes is set. According to the actual application of the LSTM+KF algorithm model in clock error prediction, the input node is set to 1 for inputting the measured value of the clock error; the intermediate nodes are set to 4; and the output nodes are set to 2 for outputting the prior knowledge in the KF algorithm, that is, the state noise variance Q t and the observation noise variance R t .

[0090] First, the LSTM+KF model training was completed. The clock error data was input into the preset algorithm model. 50 rounds of iterative training were performed to achieve model convergence. The error curve of the iterative process is shown in the figure below. Figure 2 As shown in the figure, the LSTM+KF model achieved convergence after about 15 iterations after 50 iterations. Therefore, the number of iterations can be appropriately reduced during subsequent algorithm model optimization to improve learning efficiency.

[0091] The same set of clock error data was estimated and predicted using a separate KF algorithm (using the empirical values ​​from practice as the prior parameters), a separate trained LSTM algorithm, and the trained LSTM+KF algorithm. The results are as follows: Figure 3 The clock error data is based on approximately 3 hours of measurement data with a 1-second interval. Since the clock error data has a deviation of approximately 90ms, this large deviation was subtracted for ease of demonstration and calculation, leaving only the clock error trend in the results.

[0092] like Figure 3 As shown in the figure, we can see that the KF algorithm can achieve good prediction results, but there is still room for optimization and improvement. This is mainly due to the influence of prior knowledge. If we want to improve the prediction ability of the KF algorithm, we need to further improve the prior parameters.

[0093] Using a separate LSTM algorithm for training and performing estimated predictions based on clock error data, we found that the prediction results deviated when the slope of the clock error data changed significantly. This was mainly because the training dataset was small and did not cover all clock error change scenarios. We can improve its prediction accuracy and prediction ability by iterating the LSTM network through collecting longer clock error data.

[0094] The LSTM+KF algorithm avoids the above problems. It not only solves the problem that the KF algorithm requires prior knowledge, but also solves the problem that the LSTM algorithm has poor model generalization ability due to the small training data set.

[0095] After training, the state noise variance Q output by the LSTM+KF model is t and the observation noise variance R t Curves such as Figure 4 As shown in the figure, the comparison between the predicted values ​​of clock error and the actual measured values ​​of clock error by three different algorithm models is as follows: Figure 5 、 Figure 6 and Figure 7 shown.

[0096] In order to more intuitively compare the results obtained by the three different algorithms, their predicted values ​​were compared with the clock error measurements. Since the LSTM algorithm model did not achieve good prediction ability in the clock error test set prediction, the variance calculation of the clock error results predicted by it was not performed. The variance of the actual measured clock error value was 1.26ns, the KF algorithm model predicted the clock error value variance to be 1.248ns, and the LSTM+KF algorithm model predicted the clock error value variance to be 1.239ns. Therefore, it can be said that the LSTM+KF algorithm model proposed in the present invention has a great application prospect in the field of clock error prediction, and provides a new solution for the correction of the clock error model of the time synchronization system.

[0097] In summary, the present invention optimizes the Kalman filter algorithm using a long short-term memory network, establishes a Kalman filter algorithm model based on the long short-term memory network, and uses this model to predict clock errors in a time synchronization system based on two-way comparison. The present invention solves the problem of the Kalman filter algorithm's reliance on prior knowledge, and combines the long-term memory capacity of the long short-term memory network with the real-time update characteristics of the Kalman filter algorithm to achieve more accurate clock error prediction. The present invention helps to improve the performance of time synchronization systems and also provides new methods and ideas for research in related fields.

[0098] It should be noted that the technical solution of the present invention is not limited to the above-mentioned specific embodiments, and all technical variations made according to the technical solution of the present invention fall within the protection scope of the present invention.

Claims

1. A clock error prediction method based on Kalman filtering combined with long short-term memory network, characterized in that: The following steps are involved: Step 1: Build an LSTM network, including input layer, hidden layer and output layer; Step 2: Construct a KF algorithm model. The first equation of the KF algorithm model is used to derive the state prediction at time t+1 from the state estimate at time t. The second equation of the KF algorithm model is used to adjust the prediction value at time t+1 to obtain the state estimate at time t+1. Step 3: Obtain the measured clock error data. The measured clock error data is measured using a time synchronization system based on two-way comparison. A portion of the measured clock error data is intercepted as training data, and the training data is merged into a data matrix with 1 row and n columns. , n is the length of training data, and normalization is performed; Step 4: Use the normalized training data to jointly train the LSTM network and KF algorithm model; Step 5: Get the clock error data in the actual scene and merge it into a data matrix with 1 row and m columns. , m is the length of clock error data, and normalization is performed; Step 6: Input the data processed in step 5 into the trained LSTM network, and use the output of the LSTM network as the parameter Q in the KF algorithm model. t and R t , after the operation of the KF algorithm model, the clock error prediction result is obtained, where Q t represents the variance of the state noise, R t Represents the variance of the observation noise.

2. The clock error prediction method based on Kalman filtering combined with long short-term memory network according to claim 1 is characterized in that: In step 1, the input layer includes 1 node, the hidden layer includes 4 nodes, and the output layer includes 2 nodes. The change relationship of each node in the LSTM network is as follows: （1） （2） （3） （4） （5） （6） Among them, x represents the input data of the current node, σ represents the sigmoid activation function, tanh is an activation function, W is the weight matrix of each gate, the subscript of W represents two corresponding parameters, b is the bias term of each gate, ◦ represents element-wise multiplication, f represents the forget gate parameter, i represents the input gate parameter, and o represents the output gate parameter. Represents the current memory unit state, c represents the current memory unit output, h represents the output value of the current node, and the subscripts t and t-1 represent two adjacent moments.

3. The clock error prediction method based on Kalman filtering combined with long short-term memory network according to claim 2 is characterized in that: The first equation of the KF algorithm model is: （7） Among them, F t Indicates the operating rules of the system itself. represents the state estimate at time t, represents the state prediction at time t+1; The second equation of the KF algorithm model is: （8） in, represents the state estimate at time t+1, Y t+1 represents the data observation value at time t+1, H t+1 represents the observation matrix, G t+1 is the Kalman gain, and its calculation formula is: （9） Among them, R t+1 represents the variance of the observation noise, It represents the prediction error of the data at time t+1 predicted by the data at time t, and its calculation method is: （10） Among them, Q t represents the variance value of the state noise, It represents the estimation error of the estimated value at time t estimated by the data at time t; The estimation error of the estimated value at time t+1 estimated by the data at time t+1 is: （11） Where I is the identity matrix.

4. The clock error prediction method based on Kalman filtering combined with long short-term memory network according to claim 3 is characterized in that: The specific method of step 4 is: Step 401: training data Input the LSTM network and use the output of the LSTM network as the parameter Q of the KF algorithm model t and R t ; Step 402: Q t and R t Two parameters are input into the KF algorithm model and calculated according to equations (7) to (11) to obtain the Kalman estimation value , which is the calculation result of formula (7); Step 403: Calculate the error term of the LSTM network according to formula (12): （12） Step 404: Feedback the error term error to the LSTM network to update the weight matrix; Step 405: Perform multiple rounds of iterations to complete the joint training of the LSTM network and the KF algorithm model.

5. The clock error prediction method based on Kalman filtering combined with long short-term memory network according to claim 3 is characterized in that: In step 6, the operation of the KF algorithm model refers to the calculation based on equations (7) to (11), and the obtained clock error prediction result is the calculation result of equation (7).

Images